Context-dependent in-silico organelle localization from label-free microscopy

Abstract

The present invention provides a system and method of labeling a biological cell by at least one processor. Embodiments include obtaining a transmitted-light microscopy patch depicting the biological cell, extracting a context vector representing biological characteristics based on the microscopy patch, processing the patch through a machine-learning encoder to generate a feature vector representing the cell in a latent feature space, employing a machine-learning transformation model to augment the feature vector based on the context vector, and applying a generative decoder model on the augmented feature vector to predict at least one organelle image, wherein the predicted organelle image imitates an organelle-specific fluorescent image of the biological cell. Embodiments may further include presenting the predicted organelle image via a user interface, enabling a human expert to determine a state of the biological cell and / or biological sample based on the transmitted-light microscopy patch.

Description

CONTEXT-DEPENDENT IN-SILICO ORGANELLE LOCALIZATION FROM LABEL- FREE MICROSCOPYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority from U.S. Application No. 63 / 718,647, titled "METHOD FOR INCORPORATING BIOLOGICAL CELL CONTEXT TO AN IN- SILICO LABELING MODEL", filed November 10, 2024, which is hereby incorporated by reference in its entirety.FIELD OF INVENTION

[0002] The present invention relates to computational microscopy and cellular imaging systems, and more particularly to a system for context-dependent in-silico localization of organelles and subcellular structures from label-free transmitted light microscopy images.BACKGROUND

[0003] As used herein, the term “organelles” may refer herein to specialized membrane-bound compartments within biological cells that perform distinct cellular functions, including, for example mitochondria and Golgi apparatus. The term “organelles” may also encompass other subcellular structures that may not be membrane-bound but serve important structural or functional roles within cells including, for example microtubules and actin filaments that form the cytoskeletal network. These organelles and subcellular structures collectively contribute to cellular organization and function through their spatial arrangement and dynamic interactions within the cellular environment.

[0004] Cellular organelles function as specialized compartments within biological cells, each performing distinct roles in cellular metabolism, protein synthesis, energy production, and structural maintenance. The spatial organization and dynamic interactions between organelles are fundamental to cellular function, with alterations in organelle arrangement often reflecting changes in cellular state, metabolic activity, or response to environmental conditions.

[0005] Traditional methods for visualizing organelle localization rely on fluorescent labeling techniques, where specific proteins or structures are tagged with fluorescent markers to enable their detection through microscopy. While fluorescent labeling provides high specificity for individual organelles, simultaneous visualization of multiple organelle types within the same cell presents technical challenges. The spectral overlap between different fluorescent markers limitsthe number of organelles that can be simultaneously observed, and the labeling process itself may introduce artifacts or alter cellular behavior.

[0006] Label-free microscopy techniques, such as brightfield and phase contrast imaging, offer alternatives for cellular observation but lack the specificity to distinguish individual organelles. These approaches capture cellular morphology and optical properties without requiring fluorescent markers, enabling long-term observation of living cells without phototoxicity concerns. However, the interpretation of label-free images for organelle identification remains challenging due to the limited contrast and specificity of these imaging modalities.

[0007] Computational approaches have emerged to bridge the gap between label-free imaging and organelle-specific visualization. Machine learning models can be trained to predict organelle localization patterns from transmitted light microscopy images by learning associations between optical features and fluorescent labeling patterns. These in-silico labeling methods enable the computational generation of organelle-specific images from label-free microscopy data, potentially allowing simultaneous visualization of multiple organelles without the limitations of fluorescent labeling.

[0008] Current in-silico labeling approaches face challenges when applied to out-of- distribution data such as cells exhibiting altered intracellular organization compared to typical cellular states represented in training datasets. For example, cells undergoing dynamic processes such as division, migration, or stress responses may display organelle arrangements that differ substantially from the predominant cellular contexts used for model training. Other examples for such out-of-distribution data may include introduction of different cell types, different microscope settings, and other perturbations that alter the label-free mapping, resulting in reduced prediction accuracy in context-indifferent.

[0009] The heterogeneity of cellular populations presents additional challenges for organelle localization prediction. Cells within the same population may exhibit diverse morphologies, metabolic states, and environmental contexts that influence organelle organization. Conventional approaches that rely solely on image-based features may not adequately account for these contextual factors that affect cellular organization patterns.

[0010] As used herein, the term "context" or "cell context" refers to characteristics, conditions, or circumstances associated with a biological cell that may affect the prediction of organelle localization from transmitted light microscopy images. Cell context encompasses a broad spectrumof information that extends beyond the optical properties directly observable in transmitted light microscopy images, providing additional knowledge that may guide accurate computational interpretation of cellular structures and organelle arrangements.

[0011] Intrinsic cellular contexts may include characteristics that relate to the cell's current biological state or physical properties. Mitotic stage context may represent the cell's position within the cell cycle, distinguishing between interphase cells and various phases of mitosis including prophase, prometaphase, metaphase, anaphase, and telophase, each characterized by distinct organelle reorganization patterns. Shape contexts may encompass geometric measurements such as cell volume, height, width, and morphological features derived through machine learning analysis of cellular boundaries. Neighborhood density context may quantify the local cellular environment by measuring the number of adjacent cells, reflecting the influence of cell-cell interactions and mechanical constraints on organelle organization.

[0012] Spatial or positional contexts may characterize the cell's location within its broader cellular environment. Location context may distinguish between cells positioned at colony edges versus those in colony interiors, where edge cells may experience different mechanical stresses, nutrient gradients, and cell-cell contact patterns that may influence organelle arrangement. These positional factors may lead to polarized cellular morphologies and asymmetric organelle distributions that differ from cells in more constrained interior positions.

[0013] Extrinsic contexts may encompass external factors and experimental conditions that may affect the computational prediction of organelle localization without necessarily altering the actual cellular organization. Cell type context may account for differences between various cellular lineages or differentiation states, each exhibiting characteristic organelle arrangements and metabolic profiles. Perturbation contexts may include experimental treatments, drug exposures, or environmental stresses that may alter cellular organization patterns. Disease state contexts may reflect pathological conditions that may systematically modify organelle localization and cellular architecture. Technical contexts may include assay conditions, microscope settings, imaging parameters, and fluorescent marker types that may influence the relationship between transmitted light microscopy appearance and organelle localization patterns, affecting computational analysis without changing the underlying cellular structure.

[0014] Current in-silico labeling approaches face challenges when applied to out-of- distribution data such as cells exhibiting altered intracellular organization compared to typicalcellular states represented in training datasets. For example, cells undergoing dynamic processes such as division, migration, or stress responses may display organelle arrangements that differ substantially from the predominant cellular contexts used for model training. Other examples for such out-of-distribution data may include introduction of different cell types, different microscope settings, and other perturbations that alter the label-free mapping, resulting in reduced prediction accuracy in context-indifferent systems. The heterogeneity of cellular populations presents additional challenges for organelle localization prediction. Cells within the same population may exhibit diverse morphologies, metabolic states, and environmental contexts that influence organelle organization. Conventional approaches that rely solely on image -based features may not adequately account for these contextual factors that affect cellular organization patterns.SUMMARY

[0015] As explained herein, context-aware analysis may enable improved prediction of organelles and subcellular structures in label-free images by incorporating contextual knowledge that guides computational interpretation of cellular optical properties, thereby addressing limitations of current technology that may exhibit reduced accuracy when applied to rare or altered cellular contexts.

[0016] Additionally, inclusion of extrinsic context descriptors could be used to harmonize datasets from multiple resources to one large dataset. Thus context-dependent in-silico labeling has the potential to be the enabler toward training general in-silico labeling “foundation models”. An exciting possibility is to integrate the cell’s continuous state during the progression of a physiological process as intrinsic context for in-silico labeling. For example, using the FUCCI system as a rich cell cycle context or by computational prediction of the continuous cell state. Notably, while cell extrinsic contexts should be available via the experimental meta data, intrinsic contexts are commonly computationally derived from the image data, thus introducing measurement errors that would affect the in-silico labeling.

[0017] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0018] The present invention relates to a system and method for context-aware, image-to- image translation. According to some embodiments, this context-dependent image-to-imagetranslation may include in-silico organelle localization from label-free transmitted light microscopy images. The system incorporates biological cell context information to enhance the accuracy of computational organelle prediction, addressing limitations of conventional approaches that may exhibit reduced performance when applied to rare cellular contexts under-represented in training datasets.

[0019] The present invention addresses the currently felt need for cost-effective and accurate prediction of organelle labeling by incorporating cellular context information to guide organelle localization predictions. This approach enables enhanced prediction accuracy for cells exhibiting altered intracellular organization due to specific cellular states, environmental conditions, or physiological processes that may not be adequately represented in training datasets. The invention's applicability extends to subsequent analysis and diagnosis of biological samples at the cellular level through improved organelle localization, shape, colocalization patterns and dynamics analysis, at the tissue level through enhanced understanding of cellular organization within tissue architecture, and at the patient level through better characterization of disease-related cellular alterations and therapeutic responses.

[0020] The present invention may provide a practical application in the field of microscopy image analysis, e.g., for improving computer-assisted diagnostic technology, by enabling accurate and comprehensive analysis of cellular organization patterns from label-free microscopy images.

[0021] The term “patch” may be used herein in reference to input of single-cell images, and may have slightly different meaning, according to context: As known in the art, deep learning models use batches to efficiently estimate gradients using multiple samples at once, balancing computational speed with more stable and reliable training updates. Embodiments of the invention (e.g., autoencoder 200, transformation module 300) may be trained using batches of 24 samples.

[0022] Since it is typically not feasible to load 24 full 3D single-cell images into a standard GPU, system 100 may be trained on patches extracted from these images. In this context, a patch 430 may be a 3D crop from the original single-cell volume, taken from both the brightfield input image 20 and target channels (e.g., fluorescent images depicting specific organelles) of multiplexed images in training dataset 700DS. According to some embodiments, during a training stage, autoencoder 200 and transformation module 300 may be trained on batches that each include 24, where the pixel dimensions of these patches are 32 x 64 x 64 along the z, y, and x axes, respectively. During a subsequent inference stage, system 100 may be loaded patches 430 thatinclude full single-cell images one by one, to produce predicted organelle images as elaborated herein.

[0023] According to some embodiments, at least one processor may be configured to obtain a transmitted-light microscopy patch depicting a biological cell. The processor may be further configured to extract a context vector representing biological characteristics of the biological cell based on the microscopy patch. In some embodiments, the processor may also be adapted to process the microscopy patch through a machine-learning based encoder to generate a feature vector representing the depicted biological cell in a latent feature space. The processor may employ a machine-learning based transformation model to augment the feature vector based on the context vector. The processor may apply a generative machine-learning based decoder model on the augmented feature vector to predict at least one organelle image, wherein the predicted organelle image may imitate an organelle-specific fluorescent image of the biological cell.

[0024] According to some embodiments, the context vector may include entries representing intrinsic cellular properties such as a cell cycle stage indicator, a spatial location indicator within a cell population, a shape feature descriptor, a morphological feature descriptor, a neighborhood density measure and any combination thereof.

[0025] Additionally, or alternatively, the context vector may include entries representing extrinsic cellular properties such as a perturbation context indicator, a condition context indicator, a cell type context indicator, an imaging parameter context indicator, a sample preparation context indicator and any combination thereof.

[0026] In some embodiments, the processor may be further configured to analyze the at least one predicted organelle image to determine a state of the biological cell. Such a state may include, for example, a cell cycle progression stage, an organelle spatial distribution pattern, a cellular stress response level, a metabolic activity state, a differentiation status, an apoptosis progression stage, an epithelial-to-mesenchymal state, progression through viral infection, a cell migration potential, a drug response phenotype, a disease pathology indicator, a tissue organization role, a cell-cell interaction status, an organelle dysfunction marker, a cellular aging indicator, and any combination thereof.

[0027] As explained herein, the at least one processor may present the at least one predicted organelle image via a user interface, enabling a human expert to determine a state of the biologicalcell and / or a state of a biological sample comprising the biological cell, based on the transmitted- light microscopy patch.

[0028] According to some embodiments, the at least one processor may employ the machinelearning based transformation model by feeding the context vector through at least one first neural network layer to obtain one or more scaling parameters. Additionally, or alternatively, the at least one processor may feed the context vector through at least one second neural network layer to obtain one or more shift parameters. In some embodiments, the processor may then generate an affine transformation using the one or more scaling parameters and shift parameters, and may apply the affine transformation to the feature vector to produce the augmented feature vector.

[0029] According to some embodiments, the at least one processor may, e.g., during a training stage, obtain a training dataset including pairs of transmitted-light microscopy patches and corresponding ground truth organelle-specific fluorescent images, each pair associated with a respective context vector. The at least one processor may process transmitted-light microscopy patches of the training set through the machine-learning based encoder to generate interim feature vectors. The at least one processor may feed each context vector through the at least one first neural network layer and the at least one second neural network layer to respectively generate interim scaling parameters and interim shift parameters. The processor may apply affine transformations to the training feature vectors using the interim scaling parameters and interim shift parameters to produce interim augmented feature vectors. According to some embodiments, the processor may then process the training augmented feature vectors through the generative machine-learning based decoder model to generate interim predicted organelle images, and train the machine-learning based transformation model based on the interim predicted organelle images.

[0030] According to some embodiments, the processor may be configured to train the machine-learning based transformation model by calculating a reconstruction error between an interim organelle image and a corresponding ground truth organelle-specific fluorescent image. The processor may, for example, use a backpropagation algorithm to update at least one weight of the at least one first neural network layer or the at least one second neural network layer so as to minimize the reconstruction error.

[0031] According to some embodiments the at least one processor may obtain the transmitted- light microscopy patch by acquiring, and processing a field-of-view transmitted-light microscopy image depicting a plurality of biological cells. The processor may perform cell segmentation onthe field-of-view image to generate segmentation masks identifying individual biological cells within the field-of-view. The processor may select a target biological cell from the plurality of biological cells, and crop a region of the field-of-view corresponding to the target biological cell based on a corresponding segmentation mask to create the transmitted-light microscopy patch while isolating the target biological cell from background regions.

[0032] As elaborated herein, the at least one processor may be configured to generate a plurality of predicted organelle images for different organelle types from the same transmitted- light microscopy patch. Each predicted organelle image may be generated using a respective (e.g., separate) machine-learning based transformation model trained for a specific organelle type. The at least one processor may subsequently create a multiplexed organelle image by combining the plurality of predicted organelle images. In some embodiments, the processor may be configured to analyze spatial relationships between different organelles in the multiplexed organelle image to determine an organelle colocalization pattern.

[0033] The organelle colocalization pattern may refer to the spatial arrangement and proximity relationships between different cellular organelles within a cell, which can indicate cellular states such as metabolic activity, stress responses, or disease conditions based on how organelles are positioned relative to each other.

[0034] The processor may be further adapted to determine a state of the depicted cell based on the organelle colocalization pattern, as elaborated herein. For example, enhanced colocalization between mitochondria and endoplasmic reticulum may indicate increased metabolic activity, while dispersed mitochondrial distribution with reduced nuclear envelope organization may suggest cellular stress or early apoptotic progression.

[0035] Additionally, or alternatively, the at least one processor may acquire a time series of transmitted-light microscopy patches depicting the biological cell at sequential time points. It may process each transmitted-light microscopy patch in the time series through the machine-learning based encoder, the ML-based transformation model, and / or the generative ML-based decoder model to generate corresponding time series of predicted organelle images. The at least one processor may subsequently combine the time series of predicted organelle images to create a livecell multiplexed image sequence enabling dynamic visualization of organelle localization patterns over time.

[0036] Additionally, or alternatively, cell analysis module 800 may be configured to analyze predicted organelle images 60 to determine organelle shape characteristics by processing spatial features and morphological parameters extracted from the predicted organelle localization patterns. The shape analysis may involve measurement of organelle dimensions, aspect ratios, surface area calculations, and geometric descriptors that characterize organelle morphology. Cell analysis module 800 may monitor changes in organelle morphology over time by processing sequential predicted organelle images 60 generated from time series of transmitted-light microscopy patches 430, enabling quantitative assessment of organelle shape variations during cellular processes such as mitosis, differentiation, or stress responses. The system may correlate observed organelle shape variations with cellular state transitions by comparing morphological measurements across different temporal points and associating shape changes with specific cellular contexts provided by context vector 500CV.

[0037] Additionally, or alternatively, cell analysis module 800 may be configured to analyze predicted organelle images 60 to quantify organelle composition within biological cells by measuring relative signal intensities, spatial distributions, and abundance patterns of different organelle types. The composition analysis may involve integration of pixel intensities across predicted organelle images 60 to determine relative organelle content and calculate compositional ratios between different cellular structures. System 100 may determine relative abundance of different organelle types by processing multiple predicted organelle images 60 generated for the same transmitted-light microscopy patch 430 using respective transformation modules 300 trained for specific organelle categories. The compositional measurements may be associated with cellular metabolic states or differentiation processes by correlating organelle abundance patterns with cellular context information encoded in context vector 500CV.

[0038] Additionally, or alternatively, cell analysis module 800 may be adapted to process time series of predicted organelle images 60 to analyze organelle dynamics by tracking movement patterns, redistribution events, and structural changes of organelles over sequential time points. The dynamic analysis may involve calculation of displacement vectors, velocity measurements, and trajectory mapping for individual organelle structures identified within predicted organelle images 60. System 100 may correlate organelle dynamic patterns with cellular process progression or environmental responses by comparing temporal organelle behavior with cellular context information, including mitotic stage indicators, perturbation contexts, and neighborhood densitymeasures provided by context extraction module 500. The dynamic tracking capabilities may enable quantitative assessment of organelle mobility, fusion events, and reorganization patterns that characterize specific cellular states and physiological processes.

[0039] Additionally, or alternatively, cell analysis module 800 may be configured to analyze spatial interactions between different organelles by processing multiple predicted organelle images 60 generated for the same transmitted-light microscopy patch 430, measuring proximity relationships, contact sites, and coordinated movements between organelle types. The interaction analysis may involve calculation of distance measurements between organelle centers of mass, overlap indices at pixel level, and correlation coefficients between different organelle spatial distributions. System 100 may quantify organelle interaction patterns using spatial correlation metrics that assess the degree of association between different organelle types within individual cells. Cell analysis module 800 may determine cellular states based on organelle interaction patterns by comparing measured interaction parameters with characteristic signatures associated with specific cellular contexts encoded in context vector 500CV.

[0040] Additionally, or alternatively, cell analysis module 800 may be adapted to monitor continuous transitions between different cellular states by analyzing temporal changes in organelle localization, shape, composition, and interactions through sequential processing of predicted organelle images 60 generated from time series data. The system may generate cellular state trajectories by tracking quantitative measurements of organelle organization parameters over time and correlating these measurements with evolving cellular contexts provided by context extraction module 500. System 100 may identify cellular state transition markers from organelle organization patterns by detecting characteristic changes in organelle arrangement, morphology, or interaction patterns that correspond to specific cellular process milestones or environmental response stages, enabling automated recognition of cellular state transitions based on organelle-derived features.

[0041] According to some embodiments, the processor may be configured to modify at least one parameter of the context vector while maintaining the transmitted-light microscopy patch unchanged to generate a modified context vector. The processor may process the transmitted-light microscopy patch and the modified context vector through the neural network model to generate an altered predicted organelle image, to be presented via a user interface. Additionally, or alternatively, the processor may be configured to compare the altered predicted organelle image with an original predicted organelle image generated using an unmodified context vector. Theprocessor may present the altered predicted organelle image and pixel-wise differences between the altered and original images on the user interface, enabling visual comparison of organelle localization patterns under different contextual conditions. This presentation may highlight significant structural changes such as mitochondrial redistribution during metabolic transitions or cytoskeletal reorganization during cell migration, providing interpretable visual feedback that allows human experts to understand how specific cellular contexts influence organelle organization patterns and cellular function.

[0042] The at least one processor may be further configured to quantify context-dependent changes in organelle organization by measuring correlation differences between the altered predicted organelle image and the original predicted organelle image. This quantification may provide objective metrics for assessing the magnitude of context-dependent cellular reorganization, enabling systematic comparison of organelle redistribution patterns across different cellular states and facilitating automated detection of significant structural alterations that may indicate specific physiological or pathological conditions.

[0043] The foregoing general description of the illustrative embodiments and the following detailed description thereof are merely exemplary aspects of the teachings of this disclosure and are not restrictive.BRIEF DESCRIPTION OF FIGURES

[0044] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0045] Fig. 1 is a block diagram depicting a computing device which may be included in an implementation of a system 100 for automatic organelle labeling and localization, according to some embodiments of the invention.

[0046] Fig. 2 is a block diagram depicting an exemplary implementation of a system for automatic organelle localization based on label-free transmitted-light microscopy images, according to some embodiments of the invention.

[0047] Figs. 3A and 3B are block diagrams that jointly depict an example of an implementation of a context extraction module which may be included in the system for automatic organelle labeling and localization, according to some embodiments of the invention.

[0048] Figs. 4A, 4B and 4C are schematic diagrams which depict an example of a system for automatic organelle localization, implementing cell-based analysis, according to some embodiments of the invention.

[0049] Fig. 5 is a flow diagram depicting steps of a method of labeling a biological cell by at least one processor, according to some embodiments of the present invention.

[0050] Figs. 6A-6E demonstrate inferior in-silico labeling for rare cell populations, in absence of cellular contextual information.

[0051] Figs. 7A-7C depict the incorporation of cell context information into the in-silico labeling models according to some embodiments of the invention.

[0052] Figs. 8A and 8B depict qualitative and quantitative assessment of CELTIC'S contribution to the in-silico labeling of rare cell populations according to some embodiments of the invention.

[0053] Figs. 9A-9C depict application-appropriate downstream analysis, predicting spindle axis location and orientation according to some embodiments of the invention.

[0054] Figs. 10A-10E show context-dependent generative in-silico labeling according to some embodiments of the invention.DETAILED DESCRIPTION

[0055] The following description sets forth exemplary aspects of the present disclosure. However, it should be appreciated that such description is not intended as a limitation on the scope of the present disclosure. Rather, the description also encompasses combinations and modifications to those exemplary aspects described herein.

[0056] Referring to Fig. 2, a block diagram depicts an exemplary implementation of a system 100 for automatic organelle localization based on label-free transmitted-light microscopy images. System 100 may receive a transmitted light microscopy image 20, which may be processed through a preprocessing module 400 including a cell segmentation module 410 and a masking module 420 to output single-cell microscopy patches 430, as elaborated herein.

[0057] The transmitted light microscopy image 20 may, for example, be a brightfield microscopy image acquired using spinning-disk confocal microscopy with a lOOx objective at 16- bit depth. In some cases, transmitted light microscopy image 20 may have a resolution of 624x924 pixels with a physical pixel size of 0.108x0.108 pm.

[0058] As shown in Fig. 2, preprocessing module 400 may be configured to process transmitted light microscopy image 20 to generate microscopy patches 430. According to some embodiments, preprocessing module 400 may normalize pixel intensities of all images, e.g., to have a mean of 0 and a standard deviation of 1, to account for variations in illumination intensity.

[0059] According to some embodiments, preprocessing module 400 may acquire a field-of- view (FOV) transmitted-light microscopy image 20 depicting a plurality of biological cells. Preprocessing module 400 may employ a segmentation module 410, to perform cell segmentation on the FOV image 20 to generate segmentations 410S identifying individual biological cells within the FOV. Preprocessing module 400 may further employ a masking module 420 to select a target biological cell from the plurality of biological cells, and crop a region of the FOV corresponding to the target biological cell based on a corresponding segmentation mask 420M, thereby creating a transmitted-light microscopy patch 430 while isolating a target biological cell of interest from background regions.

[0060] Cell segmentation module 410 may use 3D integer label maps where each cell's unique index is located in the label map. Cell segmentation module 410 may crop the field of view to include only pixels corresponding to the identified cell with all other pixels set to 0. In some embodiments, segmentation module 410 may exclude cells that are not entirely inside the field of view due to lack of metadata and inability to reliably extract shape descriptors.

[0061] With continued reference to Fig. 2, system 100 may include an autoencoder 200 consisting of a machine-learning (ML) based encoder module 210 and an ML-based decoder module 230. As explained herein, autoencoder 200 may be configured to process the microscopy patches 430 generated by preprocessing module 400 to produce organelle-specific predictions, also referred to herein as organelle images 60.

[0062] Encoder module 210 may be configured to process single-cell microscopy patches 430 to generate a feature vector 220FV representing the depicted biological cell in a latent feature space. In some embodiments, encoder module 210 may be implemented, for example, as a 4- level U-Net architecture with convolutional layers, batch normalization, and ReLU activations. In such embodiments, encoder module 210 may progressively down sample the input microscopy patches 430 through multiple convolutional layers to extract hierarchical features of the biological cells.

[0063] As shown in Fig. 2, encoder module 210 may generate latent feature vectors 220FV in a bottleneck layer 220. Bottleneck layer 220 may be regarded as a compressed representation of microscopy patches 430 in the latent feature space. Latent feature vectors 220FV may encode spatial and morphological characteristics of the biological cells depicted in the microscopy patches 430. In other words, latent feature vectors 220FV generated by the encoder module 210 may capture structural patterns and optical properties of the depicted biological cells that are relevant for predicting organelle localization patterns.

[0064] According to some embodiments, bottleneck layer 220 may serve as an interface point where contextual information may be incorporated, or injected into the feature representation, as explained herein.

[0065] Reference is also made to Fig. 3 A and 3B, which are block diagrams depicting an example for implementation of a context extraction module 500 which may be included in system 100, according to some embodiments of the invention.

[0066] Context extraction module 500 may be configured to extract context vectors 500CV representing biological characteristics of the biological cells based on the microscopy patches 430. As explained herein, context extraction module 500 may process microscopy patches 430 independent of encoder 210, to generate a context vector 500CV. Context vector 500CV may encode contextual information about the depicted biological cells. This contextual information may include, for example, cell cycle stage, spatial location within cell populations, shape features of the depicted cell, morphological features of the depicted cell, and neighborhood features (e.g., neighborhood density measures) surrounding the depicted cell.

[0067] As explained herein, context vector 500CV may be, or may include a 16-dimensional feature vector comprising various context types (e.g., cell type, stage, location, shape, morphology, neighborhood, perturbation, etc.) concatenated together. Context vector 500CV may be regarded as representing biologically-meaningful cell context that can guide the organelle localization predictions by providing additional information about the cellular state and environment that may not be directly observable from the transmitted light microscopy images alone.

[0068] As shown in Fig. 2, system 100 may further include a transformation module 300, which may be implemented as a Dynamic Affine Feature Map Transform (DAFT) module 300. The terms DAFT module 300 and transformation module 300 may be used herein interchangeably.As explained herein, DAFT module 300 may be configured to augment the latent feature vectors 220FV based on the context vector 500CV.

[0069] In some embodiments, DAFT module 300 may be configured to calculate transformation parameters 310TP such as scaling parameters and shifting parameters based on the context vector 500CV to modify the feature representation in the bottleneck layer 220. For example, DAFT module 300 may feed context vector 500CV through at least one first Neural Network (NN) NN1 to obtain one or more first transformation parameters 310TP, including at least one scaling parameter (denoted 310TP1). Additionally, or alternatively, DAFT module 300 may feed context vector 500CV through at least one second Neural Network NN2, to obtain one or more second transformation parameters 310TP, including at least one shift parameter (denoted 310TP2).

[0070] According to some embodiments, transformation parameters 310TP may be applied to implement a scaler and a shifter, where the scaling parameters 310TP1 may be activated by a sigmoid function to generate multiplicative factors that modulate feature map amplitudes, and the shift parameters 310TP2 may provide additive offsets to adjust feature map baselines. In some cases, half of the transformation parameters 310TP may be used for scaling operations and the other half for shifting the feature map values.

[0071] According to some embodiments, transformation parameters 310TP generated by DAFT module 300 may represent an affine transformation using the one or more scaling parameters 310TP1 and shift parameters 310TP2, to apply the affine transformation to the latent feature vectors 220FV. Transformation module 300 may thereby produce augmented feature vectors 220FV (denoted 220FVA). In other words, the affine transformation of module 300 may interface bottleneck layer 220, and modify feature vector representation 220FV in the bottleneck layer 220, to incorporate the contextual information encoded in the context vector 500CV, thereby enabling context-aware organelle localization predictions.

[0072] With continued reference to Fig. 2, decoder module 230 may be implemented as a generative, ML-based decoder model that processes the augmented feature vectors 220FVA to predict organelle images 60. Decoder module 230 may be trained to generate at least one organelle image 60 based on the augmented latent feature vector 220FVA of bottleneck layer 220. In some cases, decoder module 230 may be initialized as the up-sampling portion of a U-Net architecturewith transposed convolutional layers, batch normalization, and ReLU activations that progressively reconstruct spatial resolution from the compressed feature representation 220FV.

[0073] System 100 may be configured to support patch-based analysis and / or cell-level analysis approaches for organelle localization prediction, providing flexibility in training methodologies and inference capabilities depending on specific application requirements and available training data characteristics.

[0074] Cell-based analysis may involve training system 100, including autoencoder 200 and transformation module 300, on a training dataset 700DS that comprises three-dimensional or multi-layered single-cell target images, where each layer of these single-cell target images depicts a respective type of organelle within the same cellular volume. During this training approach, the unified system may learn to generate comprehensive cellular representations that encompass multiple organelle types simultaneously. During subsequent inference operations, decoder module 230 may produce organelle images 60 as complete three-dimensional single-cell images that are multiplexed or multi-layered, enabling simultaneous visualization of multiple organelle types within individual predicted images without requiring separate processing steps for each organelle category.

[0075] Reference is also made to Figs. 4A, 4B and 4C, which depict an example of system 100, implementing cell-based analysis, according to some embodiments of the invention. As shown in Figs. 4A, 4B and 4C, system 100 may be configured as a unified multi-organelle model where organelle type may be provided as context through one -hot encoding representation.

[0076] Fig. 4A depicts dataset unification, where field-of-view images corresponding to multiple organelle types may be pooled to create a combined training dataset 700DS. Each field- of-view image may be associated with a one-hot vector indicating the target organelle type for prediction.

[0077] Fig. 4B illustrates a training process, where system 100 may receive a training dataset 700DS that may include transmitted light microscopy images 20 along with their corresponding fluorescent target images, and one-hot vectors encoding organelle context information. System 100 (e.g., autoencoder 200 and / or transformation module 300) may be trained to minimize reconstruction error 700ER between predicted organelle images and target fluorescence images.

[0078] Fig. 4C demonstrates operation of system 100 during subsequent inference, where the trained system may receive a transmitted light microscopy image 20 and a specified organellecontext as input, and may generate a corresponding predicted organelle image 60 based on the provided contextual specification.

[0079] Patch-based analysis may involve training a plurality of instances of encoder modules 210, decoder modules 230, and / or transformation modules 300 on a training dataset 700DS that includes organelle-specific target images, where each model instance is trained in relation to a specific organelle type. During this training approach, separate neural network models may be optimized for individual organelle localization tasks, enabling specialized feature learning and prediction capabilities for each organelle category. During subsequent inference operations, each organelle-specific decoder module 230 may produce organelle images 60 that are dedicated to respective organelle types, e.g., generating separate predictions for microtubules, nuclear envelope, endoplasmic reticulum, mitochondria, and other cellular structures. These organellespecific images 60 may subsequently be processed by cell analysis module 800 to create multiplexed organelle images 800MX that register and combine multiple organelle predictions into comprehensive cellular visualizations for downstream analysis and interpretation.

[0080] An organelle image 60 generated by decoder module 230 may be regarded as an imitation of an organelle-specific fluorescent image of the biological cell depicted in the corresponding microscopy patch 430. Predicted organelle image 60 may represent a computational prediction of fluorescent labeling patterns that would be observed if the biological cell were labeled with fluorescent markers targeting specific, respective cellular organelles. In some embodiments, organelle image 60 may be generated as a 3D image stack matching the dimensions and resolution of the input microscopy patch 430.

[0081] According to some embodiments, organelle image 60 may represent multiple specific organelles. For example, the organelle image 60 may represent alpha-tubulin for microtubules, beta-actin for actin filaments, Lamin Bl for nuclear envelope, sec61B for endoplasmic reticulum, STGAL1 for Golgi apparatus, Tom20 for mitochondria, and the like. Each organelle type may correspond to a different decoder module 230 trained specifically for that organelle. Additionally, or alternatively, a single decoder module 230 may be configured to generate organelle images 60 for multiple organelle types based on additional context information specifying the target organelle.

[0082] As shown in Fig. 1, system 100 may include memory 4 storing instructions that, when executed by a processor 2, cause system 100 to generate the predicted organelle images 60 from the transmitted-light microscopy patches 430 and the context vectors 500CV.

[0083] Memory 4 may store the trained parameters of encoder module 210, decoder module 230, and transformation module 300, along with preprocessing 400 algorithms and context extraction 500 routines. During operation, processor 2 may execute these instructions to coordinate the processing pipeline from input microscopy patches 430 through context extraction 500, feature encoding 210, context-based augmentation 220FVA, and final organelle image 60 prediction.

[0084] The predicted organelle images 60 generated by decoder module 230 may provide spatial localization patterns of cellular organelles without requiring fluorescent labeling of the biological cells. This approach may enable multiplexed organelle imaging from single transmitted- light microscopy images, allowing simultaneous visualization of multiple organelle types that would otherwise require separate fluorescent markers and imaging sessions.

[0085] The present invention's applicability may extend to subsequent analysis and diagnosis of biological samples at the cellular level through improved organelle colocalization pattern analysis, at the tissue level through enhanced understanding of cellular organization within tissue architecture, and at the patient level through better characterization of disease-related cellular alterations and therapeutic responses. For example, embodiments of the invention may display predicted organelle images 60 via a user interface 60UI (e.g., output device 8 of Fig. 1) to a human expert. This human expert may thereby examine predicted organelle images 60, to conclude a state of the depicted cells and / or corresponding tissue.

[0086] As known in the art, static biological samples can be multiplexed by applying multiple fluorescent dyes to the same specimen, enabling simultaneous visualization of different organelles within a single composite image. However, live cell imaging obviously cannot employ repeated dyeing procedures. Embodiments of the present invention may overcome this limitation by digitally generating predicted multiplexed organelle images from label-free transmitted-light microscopy without requiring any fluorescent markers. In other words, embodiments of the invention may enable multiplexed live cell imaging through computational prediction rather than chemical labeling.

[0087] As shown in Fig. 2, the process may involve preprocessing module 400 acquiring sequential transmitted-light microscopy images 20 at different time points. Context extractionmodule 500 may extract context vectors 500CV for each time point, while encoder module 210 may processes each microscopy patch 430 through the latent feature space to generate sequential feature vectors 220FV in bottleneck layer 220. Transformation module 300 may augment these feature vectors based on the respective context vectors 500CV, and decoder module 230 may generate corresponding predicted organelle images 60 for each time point. The sequential predicted organelle images 60 may be combined to create predicted live-cell multiplexed images 60LC, which may, for example, be presented as dynamic video sequences via user interface 60UI, showing multiple organelle types simultaneously over time.This live-cell multiplexed representation 60LC may provide significant advantages for experts in the field by enabling real-time observation of multiple organelle dynamics simultaneously without the limitations of fluorescent labeling. The dynamic visualization may facilitate comprehensive analysis of coordinated organelle behaviors, e.g., during cellular processes (e.g., mitosis), in response to stress, and the like. Embodiments of the invention may therefore provide integrated insight into cellular organization that would be impossible to achieve through conventional fluorescent imaging approaches in living cells.

[0088] Additionally, or alternatively, embodiments of the invention may employ assistive analysis or diagnosis methods, presenting automatically generated predictions via UI 60UI for human observation.

[0089] For example, in the cellular level, embodiments of the invention may enable a human expert to identify abnormal mitochondrial fragmentation patterns that indicate early stages of apoptosis or cellular stress responses. In another example, at the tissue or patient level, embodiments of the invention may perform systematic analysis of organelle organization patterns across multiple cells, to provide machine-learning based diagnosis of neurodegenerative diseases where characteristic alterations in endoplasmic reticulum and Golgi apparatus organization serve as pathological markers for conditions such as Alzheimer's disease or Parkinson's disease, subsequently highlighting these diagnoses to a human expert via UI 60UI.

[0090] With continued reference to Fig. 2, system 100 may include a training module 700 configured to train encoder module 210, decoder module 230, and transformation module 300 based on interim output organelle images 60. Training module 700 may coordinate a supervised learning process that enables system 100 to generate accurate organelle localization predictions 60 from transmitted light microscopy images 20.

[0091] According to some embodiments, training module 700 may obtain a training dataset 700DS comprising pairs of transmitted-light microscopy patches 430 and corresponding ground truth organelle-specific fluorescent images, where each pair may be associated with a respective context vector 500CV.

[0092] The training dataset 700DS may include diverse examples of cellular contexts and their corresponding organelle localization patterns. For instance, a context vector 500CV may represent a cell in the prometaphase stage of mitosis, characterized by nuclear envelope breakdown and chromosome condensation. The corresponding ground truth organelle-specific fluorescent image of training dataset 700DS may show the characteristic reorganization of the endoplasmic reticulum into dispersed tubular networks and the formation of mitotic spindle apparatus with microtubules radiating from centrosomes toward the condensed chromosomes.

[0093] In another example, a context vector 500CV may represent a cell located at the edge of a cell colony with high shape irregularity and low neighborhood density. The associated ground truth fluorescent image of training dataset 700DS may demonstrate polarized actin filament organization with stress fibers concentrated at the cell periphery and enhanced focal adhesion formation at the leading edge, reflecting the cell's migratory phenotype.

[0094] Additionally, or alternatively, training dataset 700DS may encompass context vectors 500CV representing cells with distinct metabolic states or differentiation stages. For example, a context vector 500CV may characterize a highly metabolically active cell with large volume and dense neighborhood environment. The corresponding ground truth organelle image of training dataset 700DS may reveal an extensive mitochondrial network with elongated, interconnected mitochondria distributed throughout the cytoplasm, along with expanded Golgi apparatus and abundant endoplasmic reticulum, indicative of high protein synthesis and energy production.

[0095] In another example, a context vector 500CV may represent a quiescent cell with small volume and round morphology. The associated ground truth fluorescent image of training dataset 700DS may show fragmented mitochondria, condensed Golgi apparatus, and reduced endoplasmic reticulum organization, reflecting the cell's low metabolic activity and reduced biosynthetic capacity.

[0096] Training module 700 may process transmitted-light microscopy patches 430 of the training set through encoder module 210 to generate interim, or temporary feature vectors 220FV in bottleneck layer 220. Training module 700 may feed each context vector 500CV through the atleast one first neural network NN 1 and at least one second neural network NN2 of transformation module 300, to respectively generate interim, or temporary scaling parameters 310TP1 and interim shift parameters 310TP2. Training module 700 may apply affine transformations to the training feature vectors using the interim scaling parameters 310TP1 and interim shift parameters 310TP2 to produce interim, or temporary augmented feature vectors 220FVA.

[0097] As shown in Fig. 2, system 100 may process the training augmented feature vectors through decoder module 230 to generate corresponding interim predicted organelle images 60. Training module 700 may subsequently train transformation module 300 based on the interim predicted organelle images 60 by calculating a reconstruction error 700ER between an interim organelle image 60 and a corresponding ground truth organelle-specific fluorescent image of training dataset 700DS. Training module 700 may then use a backpropagation algorithm to update at least one weight of the at least one first neural network NN 1 or the at least one second neural network NN2 of transformation module 300 to minimize the reconstruction error 700ER.

[0098] In some embodiments, training module 700 may use mean squared error as a loss function for calculating error 700ER on the masked signal area, as known in the art.

[0099] Additionally, or alternatively, training module 700 may perform an ablation study by shuffling context vector values 500CV across the single cell population and measuring reduction in organelle localization performance with shuffled context. Training module 700 may repeat this process using different random seeds to systematically assess the contribution of context vector 500CV to the organelle localization predictions. This ablation analysis may enable training module 700 to validate the effectiveness of incorporating contextual information into the feature representation 220FVA during the training process.

[0100] With continued reference to Fig. 2, system 100 may include a cell analysis module 800 configured to process the predicted organelle images 60 to determine colocalization patterns 810 of organelles in cells depicted in transmitted light microscopy images 20. Colocalization patterns 810 may include quantitative measurements representing, for example, spatial proximity metrics between organelles (e.g., mitochondria clustering near the endoplasmic reticulum), overlapping fluorescence intensities at specific cellular locations, coordinated distribution patterns (e.g., Golgi apparatus positioning relative to the nucleus), contact sites between different organelle types, segregation patterns where certain organelles avoid specific cellular regions, and the like. For example, the quantitative measurements of colocalization patterns 810 may include Pearsoncorrelation coefficients between different organelle images, distance measurements between organelle centers of mass, overlap indices, and intensity correlation values at pixel level.

[0101] These measurements provide numerical representations of the colocalization patterns 810, enabling the system to quantify how organelles are spatially organized relative to each other. For example, a high correlation coefficient between mitochondria and endoplasmic reticulum images would indicate strong colocalization, while low correlation would suggest spatial segregation. These quantitative metrics may enable system 100 to determine cellular states such as metabolic activity levels, stress responses, or cell cycle stages based on characteristic organelle arrangement patterns 810.

[0102] Additionally, or alternatively, cell analysis module 800 may calculate colocalization patterns 810 (e.g., Pearson correlation coefficient) between fluorescent ground truth and in-silico prediction 60 for each cell, considering only pixels belonging to the cell as defined by the segmentation mask generated by cell segmentation module 410. This correlation analysis may enable cell analysis module 800 to quantify the accuracy of the organelle localization predictions 60 compared to experimentally obtained fluorescent images, providing further feedback for training module 700, in calculating error 700ER.

[0103] Inventors have demonstrated identification of spindle axis location and orientation during mitosis based on transmitted light images 20: In some embodiments, cell analysis module 800 may be configured to determine spindle axis location and orientation during mitosis by performing threshold-based segmentation at a predetermined pixel intensity percentile of the predicted organelle images 60. Cell analysis module 800 may select the two largest connected regions in the image, calculate their centers of mass, and define the spindle axis as the line connecting the two centers. In some cases, preprocessing module 400 may perform erosion of the microtubules’ prediction according to the cell's segmentation mask 430 with a predetermined kernel size, to remove residual predictions on the cell's border before analysis by cell analysis module 800. Additionally, cell analysis module 800 may measure spindle axis location error as the distance between centers of predicted and ground truth spindle axes, and orientation error as the angle between the two lines. Cell analysis module 800 may thereby enable quantitative assessment of mitotic spindle organization from transmitted light microscopy images 20 without requiring fluorescent labeling.

[0104] According to some embodiments, colocalization pattern 810 analysis may involve generating a plurality of predicted organelle images 60 for different organelle types from the same transmitted-light microscopy patch 430. Each predicted organelle image 60 may be generated using a respective transformation module 300 trained for a specific organelle type. Cell analysis module 800 may create a multiplexed organelle image 800MX by combining the plurality of predicted organelle images 60, and analyze spatial relationships between different organelles in the multiplexed organelle image 800MX to determine organelle colocalization pattern 810.

[0105] The multiplexed organelle image 800MX may be presented on a user interface 60UI with different organelle types displayed in distinct colors or channels, enabling human experts to visually assess spatial relationships, overlapping regions, and proximity patterns between multiple organelles simultaneously within a single composite image for comprehensive cellular analysis.

[0106] Cell analysis module 800 may determine a state 820ST of the depicted cell based on the organelle colocalization pattern 810. The determined state 820ST may, for example, be selected from a list consisting of cell cycle progression stage, mitotic spindle orientation, organelle spatial distribution pattern, cellular stress response level, metabolic activity state, differentiation status, apoptosis progression stage, cell migration potential, drug response phenotype, disease pathology indicator, tissue organization role, cell-cell interaction status, organelle dysfunction marker, cellular aging indicator, and any combination thereof.

[0107] The determined state 820ST may be presented on a user interface 60UI as visual overlays on the predicted organelle images 60, numerical confidence scores, or categorical classifications with associated probability distributions, enabling human experts to review and validate the automated cellular state assessments for diagnostic or research purposes.

[0108] According to some embodiments, system 100 may be configured to modify at least one parameter of context vector 500CV while maintaining the transmitted-light microscopy patch 430 unchanged to generate a modified context vector 500CVM. System 100 may process the transmitted-light microscopy patch 430 and the modified context vector 500CVM through encoder module 210, transformation module 300, and decoder module 230 as explained herein, to generate an altered version of predicted organelle image 60. Cell analysis module 800 may subsequently compare the altered predicted organelle image 60 with the original predicted organelle image 60 (generated using an unmodified context vector 500CV), and quantify context-dependent changes in organelle organization as manifested in colocalization patterns 810, e.g., by measuringcorrelation differences between the altered predicted organelle image 60 and the original predicted organelle image 60.

[0109] Reference is now made to Fig. 3A and Fig. 3B, which are block diagrams depicting exemplary implementations of context extraction module 500 that may be included in system 100. As explained herein, context extraction module 500 may be configured to process single-cell microscopy patches 430 to generate respective context vectors 500CV through multiple parallel processing paths that analyze different aspects of cells depicted in patches 430.

[0110] The multiple parallel processing paths of context extraction module 500 may enable comprehensive characterization of biological cells by extracting diverse contextual information simultaneously. Each processing path may focus on a specific aspect of cellular characteristics, and the outputs from all processing paths may be combined to form context vector 500CV representing aggregated contextual information extracted from input single-cell microscopy patches 430.

[0111] As shown in Fig. 3A, context extraction module 500 may include a cell stage processing path that generates a stage context indicator 510C representing a stage of the biological cell such as a cell cycle progression stage. The cell stage processing path may include a cell stage classifier 510 configured to analyze microscopy patches 430 and determine the mitotic stage of depicted cells. During a training stage, the cell stage classifier 510 may be trained using groundtruth cell stage annotations 510AN to recognize characteristic morphological features associated with different cell cycle phases.

[0112] Context extraction module 500 may further include a location processing path configured to produce a location context characterizing a location of specific cells within their respective cell colonies. The location processing path may include a colony edge detection module 520ED configured to identify edges that define cell colonies depicted in image(s) 20, which include the cells of microscopy patch(es) 430. A location classifier module 520 may process the identified edges to generate a location context indicator 520C that characterizes cell location in relation to identified cell colonies. For example, location context indicator 520C may include numerical values which distinguish between cells that are located at, or around colony edges versus cells positioned in colony interiors or beyond colony edges. Additionally, or alternatively, location context indicator 520C may include numerical values which define a cell's distance in relation to its respective colony.

[0113] Additionally, or alternatively, context extraction module 500 may include a shape metrics processing path that may contain a shape metrics module 530. Shape metrics module 530 may be adapted to extract metrics 530M of a cell's shape from respective single-cell microscopy patches 430. The shape metrics 530M may include measurements such as cell diameter, roundness, spiculation, height, minimum width, maximum width, and volume.

[0114] The shape metrics processing path may further include a shape clustering module 530CM, configured to cluster the shape metrics 530M to a plurality of clusters 530CL in a clustering model. Shape clustering module 530CM may associate incident cells, depicted in incoming microscopy patches 430 to respective clusters 530CL in the clustering model. Shape clustering module 530CM may subsequently generate, for each cell (each incoming single-cell microscopy patch 430) a shape context indicator 530C to characterize the depicted cells according to their resemblance in shape to other cells in unique cell groups or clusters 530CL of the clustering model.

[0115] Additionally, or alternatively, context extraction module 500 may include a morphology processing path configured to generate a morphology context indicator 540C through machine-learning based analysis. Morphology context indicator 540C is also referred to herein as machine-learning based shape context.

[0116] The morphology processing path may include a morphology classifier module 540 configured to generate embeddings or representations 540E of morphological features of cells depicted in single-cell microscopy patches 430. These morphological embeddings 540E may represent latent features of cells as perceived by the machine-learning based morphology classifier 540 rather than directly measurable characteristics. A morphology clustering module 540CM may cluster embeddings 540E to a plurality of clusters 540CL in a clustering model, and associate incident cells (cells depicted in incoming single-cell microscopy patches 430) to respective clusters 540CL in the morphology clustering model 540CM.

[0117] Additionally, or alternatively, context extraction module 500 may include a neighborhood analysis processing path or module 550 configured to generate a neighborhood context indicator 550C representing a number, or a density of cells that are adjacent to specific cells depicted in single-cell microscopy patches 430. The neighborhood analysis module 550 may quantify local cellular environment characteristics that may influence organelle organization and cellular behavior patterns.

[0118] Context extraction module 500 may further include a perturbation analysis processing path configured to generate a perturbation context indicator 560C representing, for example, treatments or interventions applied to biological cells depicted in microscopy patches 430. The perturbation context indicator 560C may characterize specific perturbations including chemical treatments, drug exposures, physical stimuli, genetic modifications, or environmental stresses that may alter cellular organization patterns and organelle localization. The perturbation context 560C may be obtained, for example, through direct user input via a graphical interface 60UI where conditions are specified. Additionally, or alternatively, perturbation context 560C may be obtained via automated detection through analysis of experimental metadata associated with imaging sessions, or through machine-learning based classification algorithms trained to identify perturbation signatures from cellular morphological changes and response patterns observable in transmitted light microscopy images.

[0119] Additionally, or alternatively, context extraction module 500 may include a cell condition analysis processing path configured to generate a condition context indicator 570C representing pathological or physiological states of biological cells that may systematically modify organelle localization and cellular architecture. The condition context indicator 570C may, for example, characterize disease states, developmental stages, or other biological conditions that influence cellular organization patterns. The condition context 570C may be obtained through direct annotation (e.g., via UI60) by medical professionals or researchers familiar with the biological samples. Additionally, or alternatively, condition context indicator 570C may be obtained via automated classification using machine-learning models trained on known disease signatures, or through analysis of clinical metadata and / or patient information associated with the cellular samples being analyzed.

[0120] Additionally, or alternatively, context extraction module 500 may include a cell type analysis processing path configured to generate a cell type context indicator 580C representing the specific cellular lineage, differentiation state, or experimental assay conditions associated with biological cells depicted in microscopy patches 430. The cell type context indicator 580C may distinguish between various cellular lineages (e.g., primary versus immortalized cell lines) or different experimental protocols that may exhibit characteristic organelle arrangements and metabolic profiles. The cell type context 580C may be obtained through direct user specification of cell line information, automated classification using machine-learning algorithms trained torecognize morphological signatures of different cell types, or through analysis of experimental protocols and sample preparation metadata associated with the imaging sessions.

[0121] Additionally, or alternatively, context extraction module 500 may include an imaging parameter analysis processing path configured to generate an imaging context indicator 590C representing technical parameters and microscope settings used during image acquisition that may influence the relationship between transmitted light microscopy appearance and organelle localization patterns. The imaging context indicator 590C may characterize microscope configurations, illumination conditions, objective lens specifications, pixel resolution, or other technical factors that affect image characteristics without altering underlying cellular structure. The imaging context 590C may be obtained through automated extraction from microscope metadata files, direct input of imaging parameters through user interfaces, or analysis of image properties that correlate with specific imaging configurations.

[0122] Additionally, or alternatively, context extraction module 500 may include a sample preparation analysis processing path configured to generate a sample preparation context indicator 595C. Sample preparation context indicator 595 may represent sample preparation parameters including, for example the usage of specific marker types, labeling protocols, and the like. The sample preparation context indicator 595C may characterize different fluorescent protein types, antibody labeling approaches, or chemical dyes that may exhibit varying binding specificities, signal intensities, or subcellular localization patterns for the same organelle types. The sample preparation context 595C may be obtained through direct specification of labeling protocols by users (e.g., via UI60) or via automated detection through analysis of fluorescence spectral properties and intensity distributions.

[0123] The outputs from all processing paths of context extraction module 500 may be aggregated to form context vector 500CV. For example, context extraction module 500 may concatenate the individual context indicators (e.g., 510C, 520C, 530C, 540C, 550C, 560C, 570C, 580C, 590C and / or 595C) directly to form a multi-dimensional context vector 500CV, where each indicator contributes specific numerical values representing different aspects of cellular characteristics. Alternatively, context extraction module 500 may generate embeddings of the individual context indicators (e.g., 510C through 595C) and apply clustering algorithms to group similar cellular contexts, with the resulting cluster assignments and distances forming the components of context vector 500CV.

[0124] Context vector 500CV may represent biological characteristics of the biological cells including cell cycle stage indicators, spatial location indicators within cell populations, shape feature descriptors, morphological ("ML-based shape") feature descriptors, neighborhood density measures, and combinations thereof. Context vector 500CV may thereby provide comprehensive contextual information that may guide organelle localization predictions by transformation module 300 as described herein.

[0125] As explained herein, the cell stage processing path may include a cell stage classifier 510 configured to generate a stage context 510C representing cell cycle progression stages of biological cells depicted in microscopy patches 430. Stage context 510C may distinguish between different phases of cell division, including interphase and various mitotic stages that exhibit distinct morphological characteristics and organelle organization patterns.

[0126] According to some embodiments, stage context 510C may be represented as a six- column one -hot vector corresponding to six cell cycle stages: M0 (interphase), M1M2 (prophase), M3 (early prometaphase), M4M5 (prometaphase / metaphase), M6M7_single, and M6M7_complete (anaphase / telophase / cytokinesis). Each stage may correspond to characteristic cellular morphologies and organelle arrangements that cell stage classifier 510 may recognize from transmitted light microscopy features.

[0127] During a training stage, cell stage classifier 510 may be trained using a cell stage annotation 510AN that provides ground truth labels for cell cycle stages. Cell stage annotation 510AN may be generated by a deep learning -based classifier combined with rule-based criteria that analyze morphological features associated with different cell cycle phases. The cell stage annotation 510AN may enable supervised learning of cell stage classifier 510 to accurately identify mitotic stages from transmitted light microscopy patterns.

[0128] In some embodiments, cell stage classifier 510 may be trained on synthetic mitotic cell images generated from interphase cells by altering their mitotic stage context through context vector 500CV manipulation. This approach may enable training module 700 to achieve classification performance without showing actual mitotic cells during the training process, thereby expanding the available training data for rare cell populations.

[0129] As explained herein, the location processing path of context extraction module 500 may include a colony edge detection module 520ED configured to identify edges defining cell colonies depicted in transmitted light microscopy image 20. Colony edge detection module 520EDmay analyze spatial arrangements of multiple cells within field-of-view images to determine boundaries that separate cell colonies from background regions or sparse cellular areas.

[0130] Colony edge detection module 520ED may process segmentation masks generated by cell segmentation module 410 to identify cells positioned at colony peripheries versus those located in colony interiors. In some embodiments, colony edge detection module 520ED may apply morphological operations and boundary detection algorithms to distinguish between cells that contact colony edges and cells surrounded by neighboring cells within colony boundaries.

[0131] As shown in Fig. 3A, the identified edges from colony edge detection module 520ED may be fed into a location classifier 520 configured to generate location context 520C. Location classifier 520 may process the edge information to characterize spatial location of specific cells depicted in single-cell microscopy patches 430 within their respective cell populations.

[0132] Location context 520C may represent a spatial location indicator within a cell population that distinguishes between cells positioned at different locations relative to colony boundaries. In some embodiments, location context 520C may be represented as a binary Boolean indicator extracted from metadata associated with the microscopy patches 430. The binary indicator may assign a first value to cells located at colony edges and a second value to cells positioned in colony interiors, thereby providing spatial context information that may influence organelle organization patterns.

[0133] Location context 520C may enable system 100 to account for spatial heterogeneity within cell populations, where cells at colony edges may exhibit different organelle arrangements compared to cells in colony interiors due to varying mechanical constraints, nutrient gradients, and cell-cell contact patterns. The spatial location indicator provided by location context 520C may thereby contribute to context vector 500CV to guide transformation module 300 in generating more accurate organelle localization predictions 60 based on cellular position within the population.

[0134] As explained herein, the shape metrics module 530 may analyze morphological characteristics of biological cells depicted in microscopy patches 430 to generate quantitative measurements that describe cellular geometry and structural features.

[0135] Shape metrics module 530 may extract shape descriptors including height, minimum width, maximum width, and volume measurements from the biological cells depicted in microscopy patches 430. In some embodiments, shape metrics module 530 may calculate cellheight as the extent along the z-axis of 3D microscopy patches 430, minimum and maximum width measurements as the smallest and largest dimensions perpendicular to the height axis, and volume as the total number of voxels occupied by the cell within the segmentation mask generated by cell segmentation module 410.

[0136] As shown in Fig. 3A, the shape metrics 530M extracted by shape metrics module 530 may be fed to a shape clustering module 530CM configured to cluster the shape metrics 530M to a plurality of clusters 530CL in a clustering model. Shape clustering module 530CM may apply k-means clustering algorithms to group cells with similar shape characteristics into distinct clusters based on their extracted shape measurements.

[0137] For example, shape clustering module 530CM may apply k-means clustering with k=5 clusters selected according to an elbow method after min-max scaling of shape measurements. The min-max scaling may normalize the shape metrics 530M to ensure that different measurement scales do not bias the clustering process, while the elbow method may determine the number of clusters by identifying the point where additional clusters provide diminishing returns in clustering performance.

[0138] Shape clustering module 530CM may associate incident cells depicted in incoming microscopy patches 430 to respective clusters 530CL in the clustering model based on similarity of their shape metrics 530M to cluster centroids. Each cluster 530CL may represent a group of cells sharing similar morphological characteristics, such as elongated cells, round cells, or cells with intermediate aspect ratios.

[0139] As further shown in Fig. 3A, shape clustering module 530CM may generate a shape context 530C for each cell to characterize the depicted cells according to their resemblance in shape to other cells in unique cell groups or clusters 530CL of the clustering model. Shape context 530C may represent cell association to respective clusters 530CL, thereby providing shape feature descriptors that capture cellular morphological characteristics.

[0140] In some embodiments, shape context 530C may be represented as a one -hot encoded vector derived from clustering of shape descriptors including height, minimum width, maximum width, and volume measurements. The one-hot encoding may assign a value of 1 to the cluster 530CL corresponding to the cell's shape classification and values of 0 to all other clusters, thereby creating a categorical representation of the cell's morphological group membership.

[0141] Shape context 530C generated by shape clustering module 530CM may contribute to context vector 500CV as a shape feature descriptor that enables transformation module 300 to account for morphological variations when generating organelle localization predictions 60.

[0142] As explained herein, the morphology processing path of context extraction module 500 may include a morphology classifier 540 configured to generate embeddings representing latent features of cell shape, or morphology that differs from directly measurable shape metrics extracted by shape metrics module 530. With continued reference to Fig. 3 A, morphology classifier 540 may be adapted to generate embeddings or representations 540E of morphological features of cells depicted in single-cell microscopy patches 430 through machine-learning based analysis rather than direct geometric measurements.

[0143] According to some embodiments, morphology classifier 540 may be implemented as an autoencoder architecture configured to process autoencoder-compressed binary cell masks that are subsequently clustered. For example, the autoencoder may consist of two 3D convolution layers with depths 16 and 32, where each convolution layer may be followed by ReLU activation and max-pooling with kernel size 2 and stride 2. The autoencoder may further include a decoder with two 3D transposed convolution layers that reconstruct the compressed representation back to the original mask dimensions.

[0144] In some embodiments, the autoencoder of morphology classifier 540 may be trained for a predetermined number of epochs to minimize mean squared error on cell segmentation images, using nearest-neighbor interpolation. The training process may enable morphology classifier 540 to learn compressed representations of cellular morphology that capture latent structural patterns not directly accessible through conventional shape measurements.

[0145] Embeddings 540E may represent latent features of cells as perceived by the machinelearning based morphology classifier 540 rather than directly measurable characteristics such as height, width, or volume. These morphological embeddings 540E may encode complex spatial patterns and structural relationships within cellular boundaries that may not be captured by traditional geometric descriptors.

[0146] As shown in Fig. 3A, embeddings 540E may be fed into morphology clustering module 540CM configured to cluster embeddings 540E to a plurality of clusters 540CL in a morphology clustering model. Morphology clustering module 540CM may apply Principal ComponentAnalysis to reduce dimensionality to a predetermined number of components, followed by K- means clustering, to group cells with similar morphological characteristics.

[0147] Morphology clustering module 540CM may associate incident cells depicted in incoming single-cell microscopy patches 430 to respective clusters 540CL in the morphology clustering model based on similarity of their embeddings 540E to cluster centroids. Each cluster 540CL may represent a group of cells sharing similar latent morphological features as determined by the machine-learning analysis of morphology classifier 540.

[0148] As further shown in Fig. 3A, morphology clustering module 540CM may generate a morphology context 540C for each cell to characterize the depicted cells according to their resemblance in morphological features to other cells in unique cell groups or clusters 540CL of the morphology clustering model. The morphology context 540C may represent specific cell association to respective clusters 540CL, thereby providing morphological feature descriptors that capture latent structural characteristics of cellular morphology.

[0149] In some embodiments, the morphology context indicator 540C may be represented as a one -hot vector based on shape cluster assignments, where each cell may be encoded according to its cluster membership in the morphology clustering model. The one-hot encoding may assign a value of 1 to the cluster 540CL corresponding to the cell's morphological classification and values of 0 to all other clusters, thereby creating a categorical representation of the cell's latent morphological group membership.

[0150] The morphology context 540C generated by morphology clustering module 540CM may contribute to context vector 500CV as a morphological feature descriptor that enables transformation module 300 to account for latent morphological variations when generating organelle localization predictions 60. The morphological feature descriptor provided by morphology context 540C may complement the shape feature descriptors from shape context 530C by capturing machine-learning derived patterns that may not be directly measurable through conventional geometric analysis.

[0151] As explained herein, neighbor analysis module 550 may be configured to generate a neighbor context 550C indicator representing a number or density of cells that are adjacent to, or in the vicinity of, specific cells depicted in single-cell microscopy patches 430. With continued reference to Fig. 3A, neighbor analysis module 550 may quantify local cellular environmentcharacteristics by analyzing spatial relationships between target cells and their surrounding cellular neighbors within transmitted light microscopy image 20.

[0152] Neighbor analysis module 550 may determine neighborhood density by counting the number of cells that are directly adjacent to or in contact with each target cell depicted in microscopy patches 430. In some embodiments, neighbor analysis module 550 may process segmentation masks generated by cell segmentation module 410 to identify neighboring cells within a predetermined distance threshold from each target cell boundary.

[0153] According to some embodiments, neighbor context 550C may be represented as a scalar quantifying local neighborhood density measured as the number of adjacent cells. The scalar value may be min-max scaled according to minimum and maximum values in a training dataset to normalize neighborhood density measurements across different cellular environments and imaging conditions. This normalization may enable consistent representation of neighborhood characteristics regardless of variations in cell colony size or cellular packing density.

[0154] In some embodiments, neighbor analysis module 550 may categorize cells into population groups based on neighborhood density measurements. For example, neighbor analysis module 550 may classify cells with fewer than a predetermined number of adjacent cells as sparse neighborhood cells, while cells with the predetermined number or more adjacent neighbors may be classified as dense neighborhood cells. The predetermined threshold may be determined based on statistical analysis of neighborhood density distributions in training datasets.

[0155] Reference is further made to Fig. 5, which is a flow diagram depicting steps of a method of labeling a biological cell by at least one processor (e.g., processor 2 of Fig. 1 ), according to some embodiments of the present invention.

[0156] According to some embodiments, the at least one processor may obtain a transmitted- light microscopy patch (e.g., 430 of Fig. 2) depicting the biological cell (step S1005). Processor 2 may do so by acquiring and preprocessing field-of-view images to isolate individual cells of interest.

[0157] At step S1010, processor 2 may extract a context vector (e.g., 500CV of Fig. 2) representing biological characteristics of the biological cell based on the microscopy patch 430. As explained herein, context vector 530CV may include intrinsic, cellular state information, spatial location information, morphological descriptors, and extrinsic information such as environmental factors and image settings.

[0158] At step S 1015, the at least one processor may process the microscopy patch through a machine-learning based encoder to generate a feature vector representing the depicted biological cell in a latent feature space, creating a compressed representation of the cellular optical properties 220FV.

[0159] At step S1020, processor 2 may employ an ML-based transformation model (300 of Fig. 2) to augment the feature vector based on the context vector, incorporating contextual information through affine transformations that modify the latent representation according to cellular characteristics, thereby producing an augmented feature vector 220FVA.

[0160] At step SI 025, processor 2 may apply a generative, ML-based decoder model (230 of Fig. 2) on augmented feature vector 220FVA to predict at least one organelle image 60. Predicted organelle image 60 may imitate an organelle-specific fluorescent image of the biological cell without requiring actual fluorescent labeling.

[0161] At step S1030, processor 2 may present the at least one predicted organelle image 60 via a user interface (60UI of Fig. 2), enabling a human expert to determine a state of the biological cell and / or a state of a biological sample comprising the biological cell based on the transmitted- light microscopy patch.

[0162] In-silico labeling of organelles may be regarded as computational cross-modality translation of label-free transmitted light microscopy images to corresponding organelle-specific fluorescent images. This computational approach may enable prediction of fluorescent labeling patterns without requiring actual fluorescent markers or multiple imaging sessions, thereby providing a non-invasive method for visualizing cellular organelles.

[0163] Organelles may act in concert to shape and enable cell function through coordinated spatial arrangements and functional interactions. The organization of organelles and spatial relations between different organelles may be versatile and can change in response to multiple factors. These factors may include cellular processes such as proliferation, migration, or differentiation, where organelles undergo systematic reorganization to support specific functional requirements.

[0164] Additionally, organelle organization may be influenced by extrinsic factors including local cell density, microenvironmental conditions such as mechanical stresses, diffusible factors, and chemical treatments. For example, during mitosis, the nuclear envelope may disassemble, thenucleus may undergo condensation and separation, the Golgi apparatus may be disassembled and reformed, and microtubules may rearrange to form the mitotic spindle apparatus.

[0165] The ability to measure alterations in intracellular organization of organelles under different experimental conditions or during physiological processes may be technically challenging due to limitations in simultaneous labeling of multiple organelles within the same cell. Conventional fluorescent labeling approaches may require separate markers for each organelle type, limiting the number of organelles that can be simultaneously visualized and potentially introducing artifacts from multiple labeling procedures.

[0166] In-silico labeling may address these limitations by enabling computationally multiplexed live cell imaging toward integrated understanding of cellular organization. The computational approach may involve acquisition of matched label-free and fluorescently labeled images, which may be used to train deep learning models that map label-free images to their corresponding matched fluorescence images. This training process may be repeated for each organelle type, producing organelle-specific in-silico labeling models.

[0167] As explained herein, during inference, organelle- specific models may be applied to generate multiplexed images displaying localization of several organelles simultaneously from single transmitted light microscopy images. This approach may enable simultaneous visualization of multiple organelle types that would otherwise require separate fluorescent markers and imaging sessions, thereby facilitating comprehensive analysis of organelle organization and interactions within individual cells.

[0168] However, changes in intracellular organization may alter cellular optical properties, potentially inducing out-of-distribution label-free images that may impair in-silico labeling performance. For example, alterations in cellular internal organization due to changes in cell-cell adhesions in densely packed microenvironments may lead to changes in cellular optical properties that can hamper high quality in-silico labeling predictions.

[0169] Context-dependent in-silico labeling approaches may address these limitations by incorporating biologically meaningful cell context information to guide organelle localization predictions. The incorporation of contextual information may enable enhanced prediction accuracy for cells exhibiting altered intracellular organization due to specific cellular states, environmental conditions, or physiological processes that may not be adequately represented in training datasets.

[0170] Out-of-distribution data caused by changes in intracellular organization across cell types, cellular processes, or perturbations may lead to altered label-free images and impaired in- silico labeling performance. These changes may occur, for example, when cells exhibit morphological or organizational characteristics that differ substantially from the cellular states represented in training datasets, resulting in optical properties that fall outside the distribution of training examples.

[0171] Rare cellular states and contexts that are under-represented in training data may pose particular challenges for in-silico labeling accuracy. Cells undergoing mitosis may represent a primary example of such rare contexts, where dramatic reorganization of cellular structures occurs during cell division. During mitotic progression, cells may exhibit substantially altered optical properties compared to interphase cells due to chromosome condensation, nuclear envelope breakdown, and reorganization of cytoskeletal networks.

[0172] The mitotic process may involve systematic changes in organelle organization that alter transmitted light microscopy appearance. For instance, during prometaphase and metaphase stages, the nuclear envelope may partially or completely disassemble, leading to dispersal of nuclear contents and altered light scattering properties. Simultaneously, the Golgi apparatus may fragment and redistribute throughout the cytoplasm, while the endoplasmic reticulum may reorganize into tubular networks with different spatial arrangements compared to interphase cells.

[0173] Microtubule organization may undergo particularly dramatic changes during mitosis, transitioning from dispersed cytoplasmic networks in interphase cells to highly organized spindle apparatus structures. These structural rearrangements may create optical signatures in transmitted light microscopy images that differ substantially from the predominantly interphase cells typically represented in training datasets, potentially leading to reduced prediction accuracy for mitotic organelle localization patterns.

[0174] Cells located at colony edges may represent another category of rare cellular contexts that can exhibit deteriorated in-silico labeling performance. Edge cells may experience different mechanical constraints, nutrient gradients, and cell-cell contact patterns compared to cells positioned in colony interiors. These environmental differences may induce alterations in cellular morphology and organelle organization that manifest as distinct optical properties in transmitted light microscopy images.

[0175] Colony edge cells may exhibit polarized morphologies with elongated shapes and asymmetric organelle distributions that reflect migratory or mechanically stressed cellular states. The reduced number of neighboring cell contacts may allow for greater morphological flexibility, leading to shape variations that may not be adequately represented in training datasets dominated by interior colony cells with more uniform morphological characteristics.

[0176] Additionally, edge cells may display altered cytoskeletal organization with enhanced stress fiber formation and modified focal adhesion patterns that can influence the spatial arrangement of associated organelles. These organizational changes may create transmitted light microscopy signatures that fall outside the typical distribution of training examples, resulting in reduced accuracy of organelle localization predictions for edge -positioned cells.

[0177] Cells with atypical volumes or those located in sparse neighborhood environments may also represent rare contexts that challenge in-silico labeling performance. Small volume cells may exhibit compressed organelle arrangements and altered spatial relationships between cellular structures, while cells in sparse neighborhoods may display morphological and organizational characteristics that differ from cells in typical density environments.

[0178] The scarcity of these rare cellular contexts in training datasets may limit the ability of conventional in-silico labeling models to accurately predict organelle localization patterns for such cells. The under-representation of mitotic cells, edge cells, and other rare populations may result in models that are optimized for the predominant cellular states present in training data, leading to systematic prediction errors when applied to cells exhibiting altered intracellular organization.

[0179] For example, experimental perturbations may introduce additional sources of out-of- distribution data that may challenge conventional in silico labeling approaches. Chemical treatments, drug exposures, or environmental stresses may alter cellular organization patterns in ways that differ substantially from untreated control conditions typically used for model training. For example, cytoskeletal disrupting agents may cause reorganization of microtubule and actin networks, while metabolic inhibitors may induce mitochondrial fragmentation and endoplasmic reticulum stress responses that may create optical signatures not represented in training datasets derived from normal physiological conditions.

[0180] Cross-database and cross-modality applications may present additional challenges for in silico labeling performance due to systematic differences in data acquisition protocols, sample preparation methods, and imaging conditions. Models trained on datasets from specificlaboratories or imaging facilities may exhibit reduced accuracy when applied to data acquired using different experimental protocols, cell culture conditions, or sample preparation techniques. These systematic differences may manifest as altered baseline optical properties, contrast variations, or modified cellular morphologies that fall outside the distribution of training examples, potentially leading to degraded organelle localization predictions.

[0181] Technical imaging parameters may also introduce substantial out-of-distribution effects that impair in silico labeling performance across different microscopy systems and acquisition settings. Variations in objective lens specifications, illumination conditions, pixel resolution, or optical configurations may create systematic differences in transmitted light microscopy appearance that affect the relationship between cellular optical properties and organelle localization patterns. Models trained on data acquired with specific microscope settings may exhibit reduced accuracy when applied to images obtained using different technical parameters, as the altered optical characteristics may not correspond to the imaging conditions represented in training datasets, resulting in systematic prediction errors for organelle localization tasks.

[0182] These limitations in handling out-of-distribution cellular contexts may restrict the applicability of in-silico labeling approaches for comprehensive analysis of cellular organization across diverse physiological states and experimental conditions. The inability to accurately predict organelle localization in rare cellular contexts may particularly impact studies investigating dynamic cellular processes, stress responses, or pathological conditions where altered cellular organization represents a characteristic feature of the biological phenomenon under investigation.

[0183] Experimental observations demonstrate decreased in-silico labeling performance for rare cell populations that are under-represented in training datasets. These rare populations include cells undergoing mitosis, cells located at colony edges, cells with small volumes, and cells positioned in sparse neighborhood environments. The deteriorated performance may be attributed to altered intracellular organization and corresponding changes in optical properties that create out-of-distribution transmitted light microscopy images.

[0184] Cells undergoing mitosis displayed declined in-silico labeling performance across all organelles compared to cells in interphase. The performance deterioration may be particularly pronounced during specific mitotic stages where dramatic structural reorganization occurs.Prometaphase and metaphase cells exhibited substantially reduced prediction accuracy, reflecting the extensive cellular reorganization that characterizes these mitotic phases.

[0185] The most dramatic performance deterioration occurred for microtubules and nuclear envelope predictions in mitotic cells. Microtubule organization undergoes extensive restructuring during cell division, transitioning from dispersed cytoplasmic networks to highly organized spindle apparatus structures. This dramatic reorganization may create transmitted light microscopy signatures that differ substantially from the interphase microtubule patterns predominantly represented in training datasets.

[0186] Nuclear envelope predictions also showed substantial accuracy reduction in mitotic cells, particularly during prometaphase and early prometaphase stages when nuclear envelope breakdown occurs. The disassembly and reformation of nuclear envelope structures during mitosis may generate optical properties that fall outside the distribution of training examples, leading to impaired prediction performance for this organelle type.

[0187] Cells located at colony edges exhibited inferior in-silico labeling performance compared to cells positioned in colony interiors. Edge-located cells demonstrated reduced prediction accuracy most notably for microtubules and Actin filaments. The altered mechanical environment and reduced cell-cell contacts experienced by edge cells may induce morphological changes and cytoskeletal reorganization that manifest as distinct optical signatures in transmitted light microscopy images.

[0188] Cells with small volumes showed decreased in-silico labeling performance for most organelles, with the most prominent reductions observed for nuclear envelope and actin filament predictions. Small-volume cells may exhibit compressed organelle arrangements and altered spatial relationships between cellular structures that create transmitted light microscopy patterns not adequately represented in training datasets dominated by cells with typical volume characteristics.

[0189] Cells positioned in sparse neighborhood environments demonstrated reduced in-silico labeling accuracy compared to cells in dense neighborhoods. The performance reduction was most substantial for nuclear envelope and Actin filament predictions. Sparse neighborhood conditions may allow for greater morphological flexibility and altered organelle organization patterns that differ from the more constrained cellular arrangements typical of dense colony environments.

[0190] The decrease in prediction quality for rare cell populations may be attributed to the application of trained models to under-represented data that exhibits out-of-distribution intracellular organization. The corresponding changes in cellular optical properties may create transmitted light microscopy signatures that fall outside the range of examples used during model training, resulting in systematic prediction errors for these rare cellular contexts.

[0191] The performance deterioration may be particularly pronounced for organelles that undergo extensive reorganization during the cellular processes or environmental conditions that define the rare populations. For example, the dramatic microtubule reorganization during mitosis may explain the substantial prediction accuracy reduction observed for mitotic cells, while the cytoskeletal changes associated with edge positioning may account for the reduced actin filament prediction performance in colony edge cells.

[0192] These experimental observations demonstrate that conventional in-silico labeling approaches may exhibit systematic limitations when applied to rare cellular contexts that are under-represented in training datasets. The deteriorated performance for rare populations may restrict the applicability of in-silico labeling for comprehensive analysis of cellular organization across diverse physiological states and experimental conditions where altered cellular organization represents a characteristic feature.

[0193] The experimental demonstration of in-silico labeling may involve a training stage and a subsequent inference stage. During training, organelle-specific models of system 100 (e.g., encoder 210, decoder 230, transformer 300) may receive label-free transmitted light images 20 (patches 430) and corresponding fluorescent target to minimize reconstruction error 700ER between model predictions and targets. During inference, each organelle-specific model may translate transmitted light images 20 to corresponding predicted fluorescence images 60, and predictions may be combined to create integrated multi-organelle images 800MX for downstream analyses.

[0194] Reference is now made to Figs. 6A-6E, which demonstrate inferior in-silico labeling for rare cell populations, in absence of cellular contextual information. Fig. 6A shows in-silico labeling. During training (left), an organelle-specific model receives label-free transmitted light images and their corresponding fluorescent targets and is trained to minimize the reconstruction error between the model’s prediction and the target. During inference (right), each organellespecific model translates a transmitted light image to its corresponding predicted fluorescenceimage. The predictions can be combined to an integrated multi-organelle image, which can be used for downstream analyses. Fig. 6B shows distribution of rare cell populations in the dataset, comprising 7,622 single cells. Left to right: Mitotic Stage: 5% of the cells were in one of five noninterphase mitotic stages; Location: 2.4% of the cells were located at the colony edge; Volume: 2.2% of the cells had z-score lower than -1.5 relative to the population distribution; Neighborhood density: 4.7% of the cells were in sparse neighborhoods consisting of 4 or less adjacent cells. Figs. 6C and 6D show cell-level predictions. Each colored region in the field of view represents the replicated U-Net’s 10 average Pearson correlation coefficient of a cell. Cells not fully contained within the field of view were masked out. Scale bar = 10pm. Fig. 6C shows poor endoplasmic reticulum in-silico labeling of three cells. Two cells were in the prometaphase / metaphase stage of mitosis, and the other cell was in a sparse neighborhood. Fig. 6D shows poor nuclear envelope in- silico labeling of two cells. One cell was in the prometaphase / metaphase and the other in the early prometaphase stages of mitosis. Fig. 6E shows distribution of single cell in-silico labeling performance across organelles for cells in interphase (dark blue) versus mitosis (light blue). Mann- Whitney U test - p-values < 0.001.

[0195] System 100 may implement context-dependent in-silico labeling by incorporating cellular contextual information through context extraction module 500 to address limitations associated with out-of-distribution cellular contexts. The context-dependent approach may enable enhanced organelle localization predictions for rare cell populations that exhibit altered intracellular organization compared to typical cellular states represented in training datasets.

[0196] The context-dependent implementation may employ a single cell centric approach that processes individual biological cells rather than field-of-view images containing multiple cells. This single cell focus may enable precise characterization of cellular contexts and targeted organelle localization predictions 60 for specific cells of interest within heterogeneous cell populations.

[0197] The single cell centric approach may include cropping single cells according to fluorescent plasma membrane-derived segmentation masks 420M generated by cell segmentation module 410 and mask module 420. The cropping process may isolate target biological cells from surrounding cellular and background regions, creating single-cell microscopy patches 430 that contain only the cellular content relevant for organelle localization analysis.

[0198] Context extraction module 500 may define five types of context representations per cell that capture different aspects of cellular characteristics and environmental conditions. These context representations may provide comprehensive characterization of biological cells that guide transformation module 300 in generating context-aware organelle localization predictions through augmented feature vectors 220FVA, as elaborated herein.

[0199] The five context representation types may be concatenated together to form context vector 500CV that provides comprehensive contextual information about biological cells. Context vector 500CV may represent a multi-dimensional feature vector where each context type contributes specific numerical values that characterize different aspects of cellular state and environment.

[0200] Context vector 500CV may enable transformation module 300 to incorporate biologically meaningful cellular context information into the feature representation process through affine transformations applied to latent feature vectors 220FV in bottleneck layer 220. The context-aware feature augmentation may guide decoder module 230 in generating organelle localization predictions 60 that account for cellular contexts and their associated organelle organization patterns.

[0201] The context-dependent approach may address limitations of conventional in-silico labeling methods that may exhibit reduced performance when applied to rare cellular contexts under-represented in training datasets. By explicitly incorporating cellular context information, system 100 may generate more accurate organelle localization predictions for cells exhibiting altered intracellular organization due to specific cellular states, environmental conditions, or physiological processes.

[0202] The single cell centric approach combined with comprehensive context characterization may enable system 100 to handle heterogeneous cell populations where individual cells may exhibit diverse cellular contexts within the same field-of-view image. This capability may facilitate analysis of dynamic cellular processes, stress responses, and experimental conditions where cellular heterogeneity represents a characteristic feature of the biological system under investigation.

[0203] As explained herein, system 100 may extend conventional autoencoder architectures by incorporating context vector 500CV into the bottleneck layer 220 through transformation module 300. This architectural extension may enable context-aware feature representation thatcombines intrinsic image characteristics with cellular contextual information to generate enhanced organelle localization predictions.

[0204] The architectural extension may build upon established U-Net frameworks by introducing context integration capabilities at the deepest layer of the network architecture. The bottleneck layer 220 may serve as an interface point where contextual information may be injected into the feature representation process, enabling the network to learn unified representations that incorporate both spatial image patterns and biological context characteristics.

[0205] Transformation module 300 may implement Dynamic Affine Feature Map Transform (DAFT) methodology to fuse image and tabular data within the autoencoder architecture. The DAFT approach may enable systematic integration of context vector 500CV with latent feature vectors 220FV through affine transformation operations that modify feature map characteristics based on cellular context information.

[0206] The affine transformation process may involve using cellular context vectors 500CV to generate scaling and shifting parameters that modify bottleneck image representations. Context vector 500CV may be processed through neural network layers within transformation module 300 to produce transformation parameters 310TP that adjust feature map amplitudes and baselines according to cellular context characteristics.

[0207] The context integration approach may enable the network to learn context-dependent feature representations that adapt to different cellular states and environmental conditions. The affine transformation operations may modulate feature map values based on cellular context, allowing the same transmitted light microscopy patterns to be interpreted differently depending on the associated cellular context information.

[0208] The unified representation generated through context integration may incorporate intrinsic image details encoded in latent feature vectors 220FV along with contextual cellular information provided by context vector 500CV. This combined representation may enable decoder module 230 to generate organelle localization predictions 60 that account for both optical characteristics of transmitted light microscopy images and biological context factors that influence organelle organization patterns.

[0209] The context-aware feature representation may address limitations of conventional approaches that rely solely on image -based features for organelle localization predictions. By incorporating cellular context information, the unified representation may enable more accuratepredictions for cells exhibiting altered intracellular organization that may not be adequately captured through image features alone.

[0210] The DAFT integration approach may enable efficient context incorporation without requiring extensive architectural modifications to established autoencoder frameworks. The transformation module 300 may be integrated at the bottleneck layer 220 while maintaining compatibility with existing encoder 210 and decoder 230 architectures, facilitating adoption of context-dependent capabilities in established in-silico labeling systems.

[0211] The context vector integration may enable the network to learn shared representations across different cellular contexts while maintaining the ability to generate context-specific organelle localization patterns. The affine transformation approach may provide a flexible mechanism for incorporating diverse types of contextual information, including categorical variables such as cell cycle stages and continuous variables such as neighborhood density measurements.

[0212] The unified representation approach may facilitate learning from limited training examples of rare cellular contexts by leveraging contextual information to guide feature interpretation. The explicit incorporation of context may enable the network to generalize organelle localization patterns across different cellular states even when specific context-organelle combinations are under-represented in training datasets.

[0213] Reference is now made to Figs. 7A-7C which depict the incorporation of cell context information into the in-silico labeling models. Fig. 7A and Fig. 7B depict context extraction. Fig. 7A shows images 20 cropped and masked to produce single-cell patches 430. Fig. 7B shows extraction of single-cell context information, represented by a 16-dimensional context feature vectors 500CV, comprising five context types (e.g., context indicators 510C through 550C of Fig. 3A).

[0214] Fig. 7C shows an exemplary architecture of system 100 according to some embodiments of the invention. This architecture may also be referred to herein as CE11 in-silico Labeling using Tabular Input Context (CELTIC).

[0215] Image patches are fed into CELTIC along with their corresponding context. The cell context is incorporated as an auxiliary input to the U-Net in-silico labeling model transforming the U-Net’s bottleneck layer (yellow) into a context-enriched feature map (green) via the DAFT 30 block (gray box with a detailed view on the right). DAFT uses its own bottleneck to fuse the imageand context, creating a scaler and shifter that adjust the feature map accordingly (see Methods and Fig. S4 for more details).

[0216] The context extraction process may involve systematic analysis of single-cell microscopy patches 430 to generate comprehensive contextual characterization through multiple parallel processing pathways. Each processing pathway may focus on specific aspects of cellular characteristics to create a multi-dimensional representation that captures diverse biological features relevant for organelle localization predictions.

[0217] A first context 510C type may represent mitotic stage information encoded as a one- hot vector corresponding to six distinct cell cycle phases. The mitotic stage context may distinguish between interphase cells and various mitotic stages including prophase, early prometaphase, prometaphase / metaphase, and anaphase / telophase / cytokinesis phases. Each mitotic stage may exhibit characteristic morphological features and organelle organization patterns that can be computationally identified from transmitted light microscopy images.

[0218] A second context 520C type may characterize spatial location of cells within their respective colonies through binary classification of edge versus interior positioning. Location context extraction may involve analysis of cell positioning relative to colony boundaries, where cells at colony peripheries may experience different mechanical constraints and cell-cell contact patterns compared to cells positioned in colony interiors. The binary location indicator may provide spatial context information that influences organelle arrangement patterns.

[0219] A third context 530C type may capture cellular shape characteristics through quantitative geometric measurements including height, minimum width, maximum width, and volume parameters. Shape context extraction may involve calculation of these geometric descriptors from three-dimensional cell segmentation masks, followed by clustering analysis to group cells with similar morphological characteristics. The clustering process may generate categorical shape classifications that represent distinct morphological phenotypes within the cell population.

[0220] A fourth context 540C type may represent machine-learning derived morphological features that differ from directly measurable geometric parameters. This morphological context may be generated through autoencoder-based analysis of cellular shape patterns, where compressed representations of cell masks may be clustered to identify latent morphologicalcharacteristics. The machine-learning approach may capture complex spatial patterns and structural relationships that may not be accessible through conventional geometric measurements.

[0221] A fifth context 550C type may quantify neighborhood density characteristics by measuring the number of adjacent cells surrounding each target cell. Neighborhood context extraction may involve analysis of spatial relationships between cells within field-of-view images to determine local cellular environment characteristics. The neighborhood density measurements may be normalized and categorized to distinguish between sparse and dense cellular environments that may influence organelle organization patterns.

[0222] These context types may be concatenated to form a 16-dimensional context feature vector 500CV that may provide comprehensive characterization of cellular state and environmental conditions. The concatenation process may combine categorical one -hot encoded vectors with continuous scalar measurements to create a unified contextual representation. The resulting context vector 500CV may encode biologically meaningful information that guides organelle localization predictions through context-aware feature augmentation.

[0223] The architectural implementation may incorporate the context vector as an auxiliary input to the autoencoder model 200 through integration at the bottleneck layer 220. The context incorporation process may transform 300 standard bottleneck representations 220FV (also referred to as feature vectors) into context-enriched feature maps 220FVA (also referred to as augmented feature vectors) that combine spatial image characteristics with cellular contextual information. This transformation 300 may enable system 100 to generate organelle localization predictions (images 60) that account for both optical properties of transmitted light microscopy images and biological context factors.

[0224] The context integration mechanism may employ affine transformation operations that use cellular context information to generate scaling 310TP1 and shifting parameters 310TP1 for feature map modification. The context vector 500CV may be processed through dedicated neural network layers to produce transformation parameters that adjust bottleneck 220 feature map amplitudes and baselines according to cellular context characteristics. The affine transformation approach may provide a systematic method for incorporating diverse types of contextual information into the feature representation process.

[0225] The transformation process may create a scaler component that generates multiplicative factors for feature map modulation and a shifter component that provides additiveoffsets for baseline adjustment. The scaler parameters may be activated through sigmoid functions to ensure appropriate scaling ranges, while the shifter parameters may provide direct additive modifications to feature map values. The combined scaling and shifting operations may enable context-dependent modulation 220FVA of feature representations 220FV.

[0226] The context-enriched feature maps generated through this transformation process may serve as input to the decoder portion 230 of the autoencoder architecture 200. The decoder may process these augmented feature representations to generate organelle localization predictions 60 that incorporate both spatial image patterns and cellular context information.

[0227] The architectural design may maintain compatibility with established autoencoder frameworks while introducing context-dependent capabilities through targeted modifications at the bottleneck layer. The context integration approach may be implemented without requiring extensive changes to encoder and decoder architectures, facilitating adoption of context-dependent functionality in existing in-silico labeling systems. The modular design may enable flexible incorporation of different types of contextual information depending on specific application requirements.

[0228] Comparative analysis between system 100 incorporating cellular context information and conventional single cell autoencoder approaches demonstrates enhanced organelle localization predictions 60 for cells exhibiting altered intracellular organization. The context-dependent implementation of the present invention may provide improved prediction accuracy for rare cellular populations that are under-represented in training datasets, particularly for organelles that undergo extensive reorganization during specific cellular processes.

[0229] The incorporation of cellular context information may contribute to enhanced predictions of endoplasmic reticulum localization patterns in dividing cells compared to context- free approaches. During mitotic progression, the endoplasmic reticulum undergoes systematic reorganization from interconnected tubular networks in interphase cells to dispersed tubular structures that facilitate cellular division processes. The context-aware approach may enable more accurate prediction of these altered endoplasmic reticulum arrangements by incorporating mitotic stage information that guides the interpretation of transmitted light microscopy patterns.

[0230] Nuclear envelope predictions may also demonstrate substantial improvement in dividing cells when cellular context information is incorporated into the organelle localization process. The nuclear envelope experiences dramatic structural changes during mitosis, includingprogressive disassembly during prometaphase and subsequent reformation during telophase. The context-dependent approach of the present invention may enable more accurate prediction of these dynamic nuclear envelope patterns by utilizing mitotic stage context to guide the interpretation of cellular optical properties associated with nuclear envelope reorganization.

[0231] The context-aware implementation of system 100 may enable prediction of astral arrays radiating from spindle poles during mitotic progression, representing a capability that may not be achievable through conventional context-free approaches. Astral arrays represent specialized microtubule structures that extend from centrosomes toward the cell periphery during mitosis, serving functions in spindle positioning and cellular division orientation. These structures may exhibit distinct optical signatures in transmitted light microscopy images that can be more accurately interpreted when cellular context information indicates mitotic progression stages.

[0232] The prediction of astral array structures may demonstrate the capability of the present invention's context-dependent approach to capture subtle organelle localization patterns that are associated with specific cellular contexts. The astral arrays may represent relatively fine structural features that require precise interpretation of transmitted light microscopy patterns in combination with contextual information about cellular division state. The successful prediction of these structures may indicate enhanced sensitivity of the context-aware approach for detecting organelle localization patterns 810 that are characteristic of particular cellular contexts.

[0233] Both conventional and context-dependent approaches may exhibit limitations in predicting complete microtubule spindle apparatus structures during mitosis, reflecting the technical challenges associated with predicting highly organized and dynamic organelle arrangements from transmitted light microscopy images. However, the context-aware implementation of the present invention may demonstrate superior performance in predicting specific components of the mitotic apparatus, such as the astral arrays that extend from spindle poles toward cellular peripheries.

[0234] The enhanced prediction capabilities for endoplasmic reticulum and nuclear envelope in dividing cells may reflect the ability of the present invention's context-dependent approach to utilize mitotic stage information to guide organelle localization predictions. The incorporation of cellular context may enable system 100 to account for the systematic organelle reorganization patterns that characterize different phases of cellular division, leading to more accurate predictions of organelle arrangements that correspond to specific mitotic stages.

[0235] The improved performance for dividing cells may be particularly significant given the under-representation of mitotic cells in typical training datasets. Mitotic cells may constitute a small fraction of total cellular populations in most experimental conditions, leading to limited training examples for these rare cellular contexts. The present invention's context-dependent approach may enable enhanced prediction accuracy for these under-represented populations by explicitly incorporating mitotic stage information that guides the interpretation of altered cellular optical properties.

[0236] The qualitative improvements observed for endoplasmic reticulum and nuclear envelope predictions in dividing cells may demonstrate the potential of context-dependent approaches to address systematic limitations of conventional in-silico labeling methods when applied to rare cellular contexts. The enhanced prediction capabilities may enable more comprehensive analysis of organelle organization during dynamic cellular processes where conventional approaches may exhibit reduced accuracy due to altered intracellular organization patterns.

[0237] Quantitative assessment of context contribution may be performed through calculation of mean contribution metrics that measure the signed difference between context-dependent and context-free organelle localization predictions. The quantitative analysis may employ delta Pearson correlation coefficient (APCC) measurements defined as the difference between context- aware predictions and conventional single cell-based predictions, where positive APCC values indicate enhanced performance through context incorporation.

[0238] The quantitative evaluation may demonstrate that cellular context enhanced prediction accuracy for cells undergoing mitosis across all organelle types analyzed. The systematic improvement observed for mitotic cells may reflect the ability of context-dependent approaches to account for the extensive intracellular reorganization that characterizes cellular division processes. The context incorporation may enable more accurate interpretation of transmitted light microscopy patterns that correspond to altered organelle arrangements during mitotic progression.

[0239] The most prominent contribution of cellular context may be measured for microtubule predictions in mitotic cells, reflecting the dramatic changes in microtubule organization that occur during cellular division. Microtubules undergo extensive structural reorganization during mitosis, transitioning from dispersed cytoplasmic networks characteristic of interphase cells to highlyorganized spindle apparatus structures that facilitate chromosome segregation and cellular division.

[0240] The dramatic microtubule reorganization during mitosis may create transmitted light microscopy signatures that differ substantially from interphase patterns, potentially leading to reduced prediction accuracy when conventional context-free approaches are applied to mitotic cells. The incorporation of mitotic stage context information may enable enhanced interpretation of these altered optical properties, resulting in improved prediction of spindle apparatus structures and associated microtubule arrangements.

[0241] The quantitative analysis may reveal substantial APCC improvements for microtubule predictions in mitotic cells compared to other organelle types, indicating that context incorporation provides particularly significant benefits for organelles that undergo extensive reorganization during specific cellular processes. The magnitude of improvement for microtubules may exceed that observed for other organelles, reflecting the correspondence between context relevance and organelle reorganization extent.

[0242] Context incorporation may also demonstrate measurable improvements for other organelle types in mitotic cells, including endoplasmic reticulum, nuclear envelope, Golgi apparatus, mitochondria, and actin filaments. The systematic enhancement across multiple organelle types may indicate that cellular context information provides broad benefits for organelle localization predictions during cellular division, even for organelles that may undergo less dramatic reorganization compared to microtubules.

[0243] The quantitative improvements for cells located at colony edges may be most notable for Golgi apparatus and microtubule predictions, reflecting the altered cellular organization patterns that may characterize edge -positioned cells. Colony edge cells may experience different mechanical constraints and reduced cell-cell contacts that influence organelle arrangement patterns, particularly for organelles involved in cellular polarity and migration processes.

[0244] Cells with small volumes may show quantitative improvement through context incorporation, particularly for endoplasmic reticulum and nuclear envelope predictions. Small volume cells may exhibit compressed organelle arrangements that create distinct transmitted light microscopy patterns, and the incorporation of volume context information may enable enhanced interpretation of these altered spatial relationships between cellular structures.

[0245] Context incorporation may enhance organelle localization predictions for cells in sparse neighborhood densities, with substantial improvements observed for Golgi apparatus and nuclear envelope predictions. Sparse neighborhood environments may allow for greater morphological flexibility and altered organelle organization patterns that differ from the constrained cellular arrangements typical of dense colony environments.

[0246] The quantitative analysis may demonstrate that context systematically contributes to organelle localization predictions across the entire cell population, although the magnitude of improvement may be marginal for the overall population due to the small fraction represented by rare cellular contexts. The systematic but modest improvement for the complete population may reflect the predominance of typical cellular states in most datasets, where context incorporation provides incremental benefits for well-represented cellular contexts.

[0247] The quantitative assessment may be performed using training datasets where rare populations constitute small fractions of total cellular examples, indicating that context-dependent approaches can learn meaningful representations from limited training data. The ability to achieve measurable improvements with minimal training examples for rare contexts may demonstrate the effectiveness of explicit context incorporation for addressing under-representation challenges in organelle localization prediction tasks.

[0248] To elucidate the contribution of each context type to organelle localization predictions for rare cell populations, an ablation study may be conducted by systematically randomizing individual context components and measuring the resulting reduction in prediction performance. The ablation analysis may involve shuffling context values across the single cell population for each context type independently while maintaining all other context components unchanged, thereby isolating the specific contribution of individual context representations to organelle localization accuracy.

[0249] The ablation methodology may involve randomizing each context type by shuffling the corresponding context values across cells within the population, effectively disrupting the association between specific cellular characteristics and their corresponding context representations. The shuffling process may be repeated multiple times using different random seeds to ensure statistical robustness of the performance reduction measurements, with the mean reduction in organelle localization accuracy serving as an indicator of each context type's contribution to prediction performance.

[0250] The ablation study may reveal that several context types contribute to organelle localization predictions for cells undergoing mitosis, with mitotic stage context ranking as the primary contributor to prediction accuracy for dividing cells. The mitotic stage context may demonstrate the largest performance reduction when randomized, indicating that explicit representation of cell cycle progression stages provides substantial guidance for organelle localization predictions during cellular division processes.

[0251] The prominence of mitotic stage context in the ablation analysis may reflect the systematic correspondence between cell cycle phases and characteristic organelle reorganization patterns. During mitotic progression, organelles undergo predictable structural changes that correspond to specific division stages, and the explicit encoding of mitotic stage information may enable the transformation module to apply appropriate context-dependent modifications to feature representations based on expected organelle arrangements for each cell cycle phase.

[0252] Additional context types may also demonstrate measurable contributions to organelle localization predictions for mitotic cells, although their individual contributions may be smaller in magnitude compared to mitotic stage context. Shape context representations may provide secondary contributions to mitotic cell predictions, potentially reflecting the morphological changes that accompany cellular division processes, including cell rounding and volume alterations that characterize different mitotic stages.

[0253] Neighborhood density context may show limited contribution to mitotic cell predictions in the ablation analysis, suggesting that local cellular environment characteristics may have minimal influence on organelle localization accuracy during cellular division compared to intrinsic cellular state information. The reduced importance of neighborhood context for mitotic cells may indicate that cell cycle-related organelle reorganization patterns are primarily determined by internal cellular processes rather than external environmental factors.

[0254] The ablation study may demonstrate differential context contributions for other rare cell populations, with location context showing primary importance for cells situated at colony edges. Edge context may demonstrate substantial performance reduction when randomized for cells located at colony peripheries, particularly for organelles involved in cellular polarity and migration processes such as endoplasmic reticulum, nuclear envelope, and microtubules.

[0255] Shape context representations may show primary contributions to organelle localization predictions for cells with small volumes, particularly for endoplasmic reticulum, Golgiapparatus, nuclear envelope, and actin filament predictions. The prominence of shape context for small volume cells may reflect the compressed organelle arrangements and altered spatial relationships that characterize cells with atypical morphological characteristics, where geometric descriptors provide guidance for interpreting altered organelle organization patterns.

[0256] The ablation analysis may reveal that machine-learning derived morphological context provides complementary contributions to conventional shape metrics, with both context types showing measurable but distinct contributions to organelle localization predictions. The machinelearning morphological context may capture latent structural patterns that are not accessible through direct geometric measurements, providing additional guidance for organelle localization predictions in cells with complex morphological characteristics.

[0257] Neighborhood density context may demonstrate variable contributions across different rare cell populations and organelle types, with some combinations showing measurable performance reductions when randomized while others exhibit minimal effects. The variable importance of neighborhood context may reflect the differential sensitivity of various organelles to local cellular environment characteristics, with some organelle types being more responsive to cell-cell contact patterns and local density conditions.

[0258] The ablation study may validate that context-dependent approaches can learn meaningful representations from limited training examples of rare cellular contexts, with measurable performance reductions observed even when specific rare populations constitute small fractions of training datasets. The ability to demonstrate context contributions through ablation analysis may indicate that explicit context incorporation enables effective utilization of sparse training examples for under-represented cellular contexts.

[0259] The systematic ablation analysis may provide quantitative validation of the biological relevance of different context types for organelle localization predictions, with context contributions corresponding to expected biological relationships between cellular characteristics and organelle organization patterns. The correspondence between ablation results and biological expectations may support the effectiveness of the context-dependent approach for incorporating biologically meaningful information into organelle localization prediction processes.

[0260] Reference is now made to Figs. 8A and 8B, which depicts qualitative and quantitative assessment of the present invention's (also referred to herein as CELTIC) contribution to the in- silico labeling of rare cell populations. Fig. 8A shows in-silico labeling visualization of mitoticcells. Each row visualizes a different organelle (top-to-bottom): endoplasmic reticulum of a cell in late mitosis, nuclear envelope of a cell in late mitosis, microtubules of a cell in the prometaphase / metaphase stage. Left-to-right: ground truth fluorescence, U-Net replicating 10, single cell-based U-Net, CELTIC, pixel-wise difference between the CELTIC model and the single-cell-based model. Red regions indicate positive intensity differences, while blue regions indicate negative intensity differences. The difference image is scaled from -1 to 1. Shown are Z slices that have been selected by an expert based on the ground truth full z-stack images. Scale bar = 5pm. Additional examples are available in Fig. S5.

[0261] Fig. 8B shows quantification of the contribution of context, via CELTIC, to the in- silico labeling of six organelles in the rare cell populations. Each marker represents the mean APCC for a given population and organelle. The marker’s color indicates the direction of the effect (blue: positive - better in-silico labeling by the inclusion of context, orange: negative - deteriorated performance after the inclusion of context); the marker’s shape encodes significance (•: not significant, ▲ : significant; Wilcoxon signed-rank test, p < 0.05); the marker’s size reflects the magnitude of the absolute difference (IAPCCI), binned into five discrete ranges.

[0262] Cell analysis module 800 may perform application-appropriate downstream analysis to determine spindle axis location and orientation during mitosis from label-free transmitted light microscopy images 20. The downstream analysis process may enable quantitative assessment of mitotic spindle organization without requiring fluorescent labeling of cellular structures. Cell analysis module 800 may process predicted organelle images 60 generated by decoder module 230 to extract spindle axis parameters that characterize cellular division processes.

[0263] The spindle axis determination process may involve threshold-based segmentation of predicted organelle images 60 at predetermined pixel intensity percentiles to identify spindle apparatus structures. Cell analysis module 800 may select the two largest connected regions within the segmented images and calculate their respective centers of mass to define geometric reference points for spindle axis characterization. The spindle axis may be defined as the line connecting the centers of mass of the two main spindle pole regions identified through the segmentation process.

[0264] Cell analysis module 800 may evaluate the accuracy of predicted spindle axis measurements using two quantitative error metrics that compare predicted parameters with ground truth spindle axis characteristics. The evaluation process may provide systematic assessment of spindle axis prediction performance for cells undergoing mitotic division processes.

[0265] Location error measurements may be calculated as the distance between the center of the predicted spindle axis line and the center of the ground truth spindle axis line. The location error metric may quantify spatial accuracy of spindle axis positioning predictions by measuring the displacement between predicted and actual spindle axis centers within the cellular coordinate system.

[0266] Orientation error measurements may be determined as the angle between the predicted spindle axis line and the ground truth spindle axis line. The orientation error metric may assess angular accuracy of spindle axis directional predictions by quantifying the rotational difference between predicted and actual spindle axis orientations. The angular measurements may enable evaluation of spindle axis alignment prediction capabilities for cells undergoing mitotic progression where spindle orientation may influence cellular division outcomes.

[0267] Reference is now made to Figs. 9A-9E which depict application-appropriate downstream analysis, predicting spindle axis location and orientation. Fig. 9A shows measurements of the predicted spindle axis. The location error (AC) denotes the distance between the centers of the predicted (pink) and the ground truth (red) spindle axes. The orientation error (A0) denotes the angle between the predicted and the ground truth spindle axes. Fig. 9B shows four representative cells (rows) in the prometaphase / metaphase mitotic stage. Columns represent (left-to-right): (i) brightfield label-free image; (ii) ground truth fluorescent microtubules image from the z-stack’s middle slice; (iii-iv) single cell-based (iii) and CELTIC (iv) in-silico labeling of microtubules; (v-vi) Threshold-based segmentation of the single cell-based (v) and CELTIC (vi) prediction. The spindle axis predictions are shown in pink, and the ground truth predictions are shown in red. In-silico labeling was performed in 3D, while spindle axis prediction and segmentation were conducted using a single 2D slice. Fig. 9C and Fig. 9D show distribution of the location (9C) and orientation (9D) errors of the single cell-based versus the present invention's (CELTIC) prediction of the spindle axis. Each data point corresponds to a single cell. Wilcoxon signed-rank test was used to reject the null hypothesis that the values with CELTIC are similar or larger than with the single cell-based model. N = 27 cells, P-value (location) < 0.00017, P-value (orientation) < 0.028. Fig. 9E shows a permutation test: CELTIC’S observed orientation error (red 'X', median AC = 13°) and the histogram of random shuffling (N=250,000). P-value < 0.00002.

[0268] System 100 may be configured as a unified multi-organelle model where organelle type may be provided as context through one -hot encoding representation. The unified modelapproach may enable training of a single system 100 to predict multiple organelle types based on contextual specification of the target organelle, rather than requiring separate models for each organelle type. The organelle context may be encoded as a multi-dimensional one -hot vector where each dimension corresponds to a specific organelle category, with a value of 1 assigned to the target organelle and values of 0 assigned to all other organelle types.

[0269] The unified model training process may involve pooling field-of-view images from a plurality (e.g., six) different organelles to create a combined training dataset that encompasses diverse organelle localization patterns within a single training framework. The pooled dataset may enable system 100 to learn shared representations across different organelle types while maintaining the ability to generate organelle-specific predictions 60 based on the provided context information 500CV.

[0270] The single system 100 may be able to predict each organelle according to the provided organelle context, where context vector 500CV includes the one-hot encoded organelle specification along with other cellular context information. During inference, the organelle type context may guide transformation module 300 to apply appropriate feature modifications that correspond to the specified organelle localization patterns, enabling decoder module 230 to generate predictions for the requested organelle type from the same transmitted light microscopy input.

[0271] The unified approach may demonstrate generalizability with new types of context beyond the original cellular context representations, indicating that the context integration framework may be extended to incorporate additional contextual information types as they become available. The flexible context integration mechanism may enable incorporation of diverse metadata types including experimental conditions, cell line identifiers, imaging parameters, or treatment conditions that may influence organelle localization patterns.System 100 may be configured to manipulate context vector 500CV while maintaining the same transmitted-light microscopy patch 430 to generate altered predicted organelle images 60 under varying contextual conditions. The context-dependent approach may enable system 100 to generate series of in-silico labeling images from identical label-free images by modifying specific parameters within context vector 500CV, thereby providing insights into context-dependent changes in cellular organization patterns.System 100 may demonstrate context-dependent generative capabilities by transitioning cells between different mitotic states, where manipulation of mitotic stage context may produce organelle images 60 showing nuclear envelope disassembly, actin filament reorganization into equatorial rings, and microtubule restructuring into mitotic spindle configurations. Additionally, system 100 may generate altered organelle localization patterns by modifying location context from interior to edge positioning, resulting in pronounced polarization of cytoplasmic structures while maintaining nuclear envelope positioning.The context-dependent generative approach may enable system 100 to overcome inter-cell variability by providing controlled traversal along context trajectories for individual cells. System 100 may quantify context-dependent changes by calculating correlation coefficients between original and altered predicted organelle images 60, with mitotic stage context producing the most dramatic alterations particularly for microtubules, nuclear envelope, and Golgi apparatus predictions. The explicit representation of cellular context may enable guided traversal along context axes, facilitating analysis of gradual changes in integrated cellular states during physiological processes.System 100 may further utilize context-dependent generative capabilities to generate synthetic training data for classification models, where manipulation of context vectors 500CV may produce sufficient synthetic mitotic cell images to train binary classifiers without requiring actual mitotic cells during training, thereby addressing data scarcity challenges for rare cellular populations.

[0272] Reference is also made to Figs. 10A-10E which show context-dependent generative in-silico labeling. Fig. 10A describes a suggested approach: the native context feature vector is modified and used to generate an altered CELTIC in-silico labeling. Fig. 10B shows in-silico labeling of a cell in interphase. The native interphase context (top) versus mitotic context (bottom). Left-to-right: actin filaments (red), nuclear envelope (green), microtubules (blue), and a multiplexed representation of all three organelles together. Shown are the central Z slices. Scale bar = 2pm. Fig. 10C shows training a mitosis classifier using interphase cells by altering their mitotic stage context, (i) Each interphase brightfield image is used twice as an input to the CELTIC microtubules model: with native M0 context and with altered M4M5 context. The resulting in- silico-labeled images are used to train a binary classifier to distinguish between the two contexts. In Fig. 10D, the mitosis classifier is evaluated on U-Net-generated in-silico images of interphaseand mitotic cells that were experimentally imaged. In Fig. 10E, AUC of the classifier on the held- out test microtubules dataset is shown.

Claims

CLAIMS1. A method of labeling a biological cell by at least one processor, the method comprising: obtaining a transmi tted-light microscopy patch, depicting the biological cell; extracting a context vector representing biological characteristics of the biological cell based on the microscopy patch; processing the microscopy patch through a machine-learning (ML) based encoder to generate a feature vector representing the depicted biological cell in a latent feature space; employing an ML-based transformation model, to augment the feature vector based on the context vector; applying a generative, ML-based decoder model on the augmented feature vector to predict at least one organelle image, wherein the predicted organelle image imitates an organelle-specific fluorescent image of the biological cell.

2. The method of claim 1 , further comprising presenting the at least one predicted organelle image via a user interface, enabling a human expert to determine a state of the biological cell and / or a state of a biological sample comprising the biological cell, based on the transmitted-light microscopy patch.

3. The method according to any one of claims 1 and 2, wherein the context vector comprises entries representing intrinsic cellular properties selected from a list consisting of: a cell cycle stage indicator, a spatial location indicator within a cell population, a cellular shape feature descriptor, a cellular morphological feature descriptor, a cellular neighborhood density measure and any combination thereof.

4. The method according to any one of claims 1-3 wherein the context vector comprises entries representing extrinsic cellular properties selected from a list consisting of: a perturbation context indicator, a condition context indicator, a cell type context indicator, an imaging parameter context indicator, a sample preparation context indicator and any combination thereof.

5. The method according to any one of claims 1-4, further comprising analyzing the at least one predicted organelle image to determine a state of the biological cell, wherein said state isselected from a list consisting of: cell cycle progression stage, mitotic spindle orientation, organelle spatial distribution pattern, cellular stress response level, metabolic activity state, differentiation status, apoptosis progression stage, cell migration potential, drug response phenotype, disease pathology indicator, tissue organization role, cell-cell interaction status, organelle dysfunction marker, cellular aging indicator, and any combination thereof.

6. The method according to any one of claims 1-5, wherein employing the ML-based transformation model comprises: feeding the context vector through at least one first neural network layer, to obtain one or more scaling parameters; feeding the context vector through at least one second neural network layer, to obtain one or more shift parameters; generating an affine transformation using the one or more scaling parameters and shift parameters; and applying the affine transformation to the feature vector to produce the augmented feature vector.

7. The method of claim 6, further comprising: obtaining a training dataset comprising pairs of transmitted-light microscopy patches and corresponding ground truth organelle-specific fluorescent images, each pair associated with a respective context vector; processing transmitted-light microscopy patches of said training set through the ML-based encoder to generate interim feature vectors; feeding each context vector through the at least one first neural network layer and the at least one second neural network layer to respectively generate interim scaling parameters and interim shift parameters; applying affine transformations to the training feature vectors using the interim scaling parameters and interim shift parameters, to produce interim augmented feature vectors; processing the training augmented feature vectors through the generative ML-based decoder model to generate interim predicted organelle images; andtraining the ML-based transformation model based on the interim predicted organelle images.

8. The method of claim 7, further comprising training the ML-based transformation model by calculating a reconstruction error between an interim organelle image and a corresponding ground truth organelle-specific fluorescent image; and using a backpropagation algorithm to update at least one weight of the at least one first neural network layer or the at least one second neural network layer, so as to minimize the reconstruction error.

9. The method according to any one of claims 1-8, wherein obtaining the transmitted-light microscopy patch comprises: acquiring a field-of-view (FOV) transmitted-light microscopy image depicting a plurality of biological cells; performing cell segmentation on the FOV image to generate segmentation masks identifying individual biological cells within the FOV; selecting a target biological cell from the plurality of biological cells; cropping a region of the FOV corresponding to the target biological cell based on a corresponding segmentation mask to create the transmitted-light microscopy patch while isolating the target biological cell from background regions.

10. The method according to any one of claims 1-9, further comprising: generating a plurality of predicted organelle images for different organelle types from the same transmitted-light microscopy patch, wherein each predicted organelle image is generated using a respective ML-based transformation model trained for a specific organelle type; creating a multiplexed organelle image by combining the plurality of predicted organelle images; analyzing spatial relationships between different organelles in the multiplexed organelle image to determine an organelle colocalization pattern; and determining a state of the depicted cell based on the organelle colocalization pattern.

11. The method according to any one of claims 1-10, further comprising: acquiring a time series of transmitted-light microscopy patches depicting the biological cell at sequential time points; processing each transmitted-light microscopy patch in the time series through the machinelearning based encoder, the ML-based transformation model, and the generative ML-based decoder model to generate corresponding time series of predicted organelle images; combining the time series of predicted organelle images to create a live-cell multiplexed image sequence enabling dynamic visualization of organelle localization patterns over time.

12. The method according to any one of claims 1-11, further comprising: modifying at least one parameter of the context vector while maintaining the transmitted- light microscopy patch unchanged to generate a modified context vector; processing the transmitted-light microscopy patch and the modified context vector through the neural network model to generate an altered predicted organelle image; comparing the altered predicted organelle image with an original predicted organelle image generated using an unmodified context vector; and quantifying context-dependent changes in organelle organization by measuring correlation differences between the altered predicted organelle image and the original predicted organelle image.

13. A system for predicting organelle localization in biological cells, the system comprising: a processor configured to obtain transmitted-light microscopy patches depicting biological cells; a context extraction module configured to extract context vectors representing biological characteristics of the biological cells; a neural network model comprising: a machine-learning (ML) based encoder configured to process the transmitted-light microscopy patches to generate feature vectors representing the depicted biological cells in a latent feature space, an ML-based transformation model configured to augment the feature vectors based on the context vectors, anda generative, ML-based decoder model configured to process the augmented feature vectors to predict organelle images that imitate organelle-specific fluorescent images of the biological cells; memory storing instructions that, when executed by the processor, cause the system to generate the predicted organelle images from the transmitted-light microscopy patches and the context vectors; and an analysis module configured to determine a state of the biological cells based on the predicted organelle images.

14. The system of claim 13, wherein the context vector comprises entries representing intrinsic cellular properties selected from a list consisting of: a cell cycle stage indicator, a spatial location indicator within a cell population, a cellular shape feature descriptor, a cellular morphological feature descriptor, a cellular neighborhood density measure and any combination thereof.

15. The system according to any one of claims 13 and 14, wherein the analysis module is further configured to analyze the predicted organelle images to determine a state of the biological cells, wherein said state is selected from a list consisting of: cell cycle progression stage, mitotic spindle orientation, organelle spatial distribution pattern, cellular stress response level, metabolic activity state, differentiation status, apoptosis progression stage, cell migration potential, drug response phenotype, disease pathology indicator, tissue organization role, cell-cell interaction status, organelle dysfunction marker, cellular aging indicator, and any combination thereof.

16. The system according to any one of claims 13-15, wherein the ML-based transformation model comprises: at least one first neural network layer configured to process the context vectors to obtain one or more scaling parameters; at least one second neural network layer configured to process the context vectors to obtain one or more shift parameters; andwherein the ML-based transformation model is configured to generate an affine transformation using the one or more scaling parameters and shift parameters, and apply the affine transformation to the feature vectors to produce the augmented feature vectors.

17. The system of claim 16, further comprising: memory storing a training dataset comprising pairs of transmitted-light microscopy patches and corresponding ground truth organelle-specific fluorescent images, each pair associated with a respective context vector; wherein the processor is configured to: process transmitted-light microscopy patches of said training dataset through the ML- based encoder to generate interim feature vectors; feed each context vector through the at least one first neural network layer and the at least one second neural network layer to respectively generate interim scaling parameters and interim shift parameters; apply affine transformations to the training feature vectors using the interim scaling parameters and interim shift parameters, to produce interim augmented feature vectors; process the training augmented feature vectors through the generative ML-based decoder model to generate interim predicted organelle images; and train the ML-based transformation model based on the interim predicted organelle images.

18. The system of claim 17, wherein the processor is further configured to train the ML-based transformation model by: calculating a reconstruction error between an interim organelle image and a corresponding ground truth organelle-specific fluorescent image; and using a backpropagation algorithm to update at least one weight of the at least one first neural network layer or the at least one second neural network layer, so as to minimize the reconstruction error.

19. The system according to any one of claims 13-18, wherein the processor is configured to obtain the transmitted-light microscopy patches by:acquiring a field-of-view (FOV) transmitted-light microscopy image depicting a plurality of biological cells; performing cell segmentation on the FOV image to generate segmentation masks identifying individual biological cells within the FOV; selecting a target biological cell from the plurality of biological cells; cropping a region of the FOV corresponding to the target biological cell based on a corresponding segmentation mask to create the transmitted-light microscopy patch while isolating the target biological cell from background regions.

20. The system according to any one of claims 13-19, wherein the neural network model comprises a plurality of organelle-specific models, each trained for a specific organelle type, and wherein the processor is configured to: generate a plurality of predicted organelle images for different organelle types from the same transmitted-light microscopy patch using respective organelle-specific models; create a multiplexed organelle image by combining the plurality of predicted organelle images; analyze spatial relationships between different organelles in the multiplexed organelle image to determine an organelle colocalization pattern; and determine a state of the depicted cells based on the organelle colocalization pattern.

21. The system according to any one of claims 13-20, wherein the processor is further configured to: modify at least one parameter of a context vector while maintaining the transmitted-light microscopy patch unchanged to generate a modified context vector; process the transmitted-light microscopy patch and the modified context vector through the neural network model to generate an altered predicted organelle image; compare the altered predicted organelle image with an original predicted organelle image generated using an unmodified context vector; and quantify context-dependent changes in organelle organization by measuring correlation differences between the altered predicted organelle image and the original predicted organelle image.

Images