Systems and methods for high-throughput single-molecule tracking in living cells

Abstract

A sequence of images visualizing the motion of molecules is received. The molecules are linked between the images. Using a variational Bayesian optimization algorithm, and based on the linking, possible trajectories of each molecule with associated probabilities are generated. Data characterizing the generated possible trajectories with associated probabilities is provided to a consuming application or process.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 476,946, filed December 22, 2022, and U.S. Provisional Application No. 63 / 476,941, filed December 22, 2022, the entire contents of each of which are incorporated herein by reference.

[0002] The subject matter described herein relates to a platform for tracking single molecules in complex systems. [Background technology]

[0003] Protein movement within the dense environment of living cells is strongly influenced by interactions with its surroundings. Single-molecule tracking (SMT) is one method for capturing protein movement as a reporter of activity. In SMT, fluorescent proteins of interest are imaged with high spatiotemporal resolution to track their movement within complex systems, such as living cells. The information embedded in these traces has been used to investigate diverse cellular phenomena, including protein-protein interactions, such as those mediating signal transduction, interorganelle communication, nuclear organization, and transcriptional regulation. However, the application of SMT technology is limited in scale and has primarily been used to address specific mechanistic hypotheses. For example, SMT is not amenable to throughput settings that enable systems-level screening or drug discovery. Summary of the Invention

[0004] In a first aspect, a sequence of images visualizing the motion of molecules is received. The molecules are linked between the images. Using a variational Bayesian optimization algorithm, possible trajectories for each molecule, with associated probabilities, are generated based on the linking. Data characterizing the generated possible trajectories, with associated probabilities, is provided to a consuming application or process.

[0005] In a correlation aspect, a sequence of images visualizing the motion of molecules is received. The molecules are linked between the images. Using a Gibbs sampling algorithm and based on the linking, possible trajectories of each molecule with associated probabilities are generated. Data characterizing the generated possible trajectories with associated probabilities is provided to a consuming application or process.

[0006] In yet another related aspect, a sequence of images visualizing the motion of molecules is received. The molecules are linked between the images. Using an adaptive hill-climbing algorithm and based on the linking, possible trajectories of each molecule with associated probabilities are generated. Data characterizing the generated possible trajectories with associated probabilities is provided to a consuming application or process.

[0007] At least a subset of the image sequence can include at least 100 molecules per image, in other variations there are at least 1000 molecules per image, and in still other variations there are at least 10,000 molecules per image.

[0008] The molecules in some variations have a density of at least 0.01 emitters per square micron per image, whereas in some variations they may have a density of at least 0.1 emitters per square micron per image.

[0009] Molecules within a biological sample can be labeled, the labeled biological sample can fluoresce, and an image sequence can be generated as the biological sample fluoresces.

[0010] Producing the image sequence can be performed using a microscope system.

[0011] Molecules can be imaged within living cells.

[0012] A probabilistic dynamic model can be inferred that includes information characterizing the trajectories of the molecules. The probabilistic dynamic model can include an array of states, and the array of states can be populated with information characterizing the trajectories of the molecules.

[0013] An internal confidence metric based on the associated probability can be generated, and the provided data can include the generated internal confidence metric. The generated internal confidence metric can be a tracking error rate lower bound that defines a lower bound on the rate of misconnections made by linking. The generated internal metric can include calculating a confidence level for each trajectory.

[0014] Furthermore, dynamic metrics can be generated independent of any particular trajectory.

[0015] Linking can include obtaining data with a plurality of statistical data extracted from the total number of detections or the number of detections in a cell.

[0016] Providing the data may include one or more of visualizing at least some of the generated possible trajectories with associated probabilities in a graphical user interface, storing at least some of the generated possible trajectories with associated probabilities in a physical persistent state, loading at least some of the generated possible trajectories with associated probabilities into memory, or transmitting at least some of the generated possible trajectories with associated probabilities over a network to a remote computing device.

[0017] At least some of the image sequences may comprise consecutive images from the corresponding moving images, while in another variation, at least some of the image sequences used to be linked are non-consecutive images from the corresponding moving images.

[0018] In another related aspect, an image sequence visualizing the movement of a molecule may be received. The image sequence may include a first type generated using a first imaging modality and a second type generated using a second, different imaging modality. Spots may be detected within the image sequence of the first type. The detected spots within the image sequence of the first type may be linked to a trajectory using a probabilistic tracking algorithm. The image sequence of the second type may be segmented to generate a plurality of instance masks. A molecule within the image sequence of the second type may be assigned to at least one instance mask of the plurality of instance masks. Data characterizing the linking and assignment may be provided to a consuming application or process.

[0019] Probabilistic tracking algorithms can take different forms, including variational Bayesian optimization algorithms, Gibbs sampling algorithms, or adaptive hill climbing algorithms.

[0020] The first imaging modality and the second imaging modality can include different molecular labeling techniques.

[0021] The first type of image sequence may be a single molecule tracking (SMT) video, and the second type of image sequence may be a non-SMT video.

[0022] The detected spots may contain intracellular components.

[0023] The types of molecules in the first type of image sequence can be labeled with distinct fluorophores.

[0024] A plurality of statistical metrics associated with at least one of the trajectories or at least one instance mask may be generated.

[0025] A hierarchy of instance masks can be stored.

[0026] The detecting may utilize one or more of a generalized log-likelihood ratio spot detector, a difference of Gaussians (DoG) detector, a Laplacian of Gaussians (LoG) detector, or a Hessian determinant (DoH) blob detector.

[0027] The detected spots can be associated with spatiotemporal coordinates using sub-pixel localization.

[0028] Sub-pixel localization can include one or more of a radially symmetric localizer, or maximum likelihood fitting to a candidate spot model using the Levenberg-Marquardt method.

[0029] In some variations, the image sequence can be generated by an apparatus for fluorescence microscopy. The apparatus can include a light source, a first optical element or assembly, a second optical element or assembly, and a detector device. The light source can be capable of emitting fluorescence excitation light. The light source can exhibit an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C. The first optical element or assembly can be configured to receive the fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam. The second optical element or assembly can include a water immersion objective configured to tilt the light beam with respect to the z-axis in the x-z plane. The second optical element can be further configured to focus the light beam on a sample surface located in the x-y plane, thereby illuminating at least a portion of the sample surface. The detector device can be configured to receive light from the illuminated portion of the sample surface. The detector device can form one or more projection images based on the light received from the illuminated portion of the sample surface.

[0030] The apparatus can include a second objective lens configured to direct light emitted from the illuminated portion of the sample surface to a detector device.

[0031] The detector device may include a semiconductor sensor.

[0032] The apparatus can include a third optical element or assembly configured to translate the light beam within the imaging plane in a direction orthogonal to the longer dimension of the light beam.

[0033] The third optical element or assembly may include a galvo mirror.

[0034] The detector device may include a solid-state sensor and may support a shutter mode for synchronizing translation of the light beam at the sample plane with selective activation or readout of the solid-state sensor.

[0035] The image sequence can be generated by a microscope system for tracking the movement of molecules. The microscope system can include a stage, a light source, a water immersion objective, and a detector device. The stage can support a sample. The sample can include molecules. The light source can emit a light beam that can induce a light-based response from the molecules in the sample. The light source can exhibit an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C. The water immersion objective can focus the light beam onto at least a portion of the sample plane. The molecules can be positioned within the sample plane. The detector device can monitor the light-based response from the molecules, which can be analyzed to track the movement of the molecules.

[0036] The microscope system may include a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction perpendicular to the longer dimension of the light beam, thereby enabling the total field of view of the microscope system in the xy plane to be expanded.

[0037] The microscope system may include a sample plane z-position controller, which allows for maintaining focus in the z-direction.

[0038] The samples can be placed in open wells of a sample plate having a plurality of open wells.

[0039] The microscope system can include an xy position controller for varying a field of view of the microscope system, the varied field of view can encompass a different subset of the plurality of open wells.

[0040] The microscope system may include a temperature controlled environment configured to control the environment of the sample plate.

[0041] Samples can be placed in open wells of a sample plate maintained at 20%-95% humidity.

[0042] Samples can be placed into open wells of a sample plate maintained at 5% CO2.

[0043] The microscope system may also include an automated sample handling robot system that enables high-throughput manipulation of multiple samples on a stage. The robot system may include a memory and a processor in communication with the memory. The robot system may also include one or more robot end effectors in communication with the processor. The one or more end effectors can manipulate the multiple samples on the stage based on communication with the processor.

[0044] Non-transitory computer program products (i.e., tangibly embodied computer program products) that store instructions, which, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform the operations described herein, are also described. Similarly, computer systems are described that may include one or more data processors and memory coupled to the one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. Furthermore, methods may be performed by one or more data processors, whether within a single computing system or distributed across two or more computing systems. Such computing systems may be connected via one or more connections, including, but not limited to, connections via a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, etc.), direct connections between one or more of the computing systems, etc., and may exchange data and / or commands or other instructions, etc.

[0045] The subject matter described herein offers many technical advantages. For example, the subject matter provides a fast, computationally efficient method for tracking targets (e.g., molecules) that separates interpretable information from background and other conflicting noise. The subject matter described herein can be used to analyze complex data that may include thousands to tens of thousands of fast-moving targets in close proximity. Furthermore, the subject matter described herein is advantageous in that it can be implemented with minimal or no human supervision.

[0046] More specifically, the present subject matter offers many technical advantages related to scalability. Current platforms are capable of generating data for over 100 molecules per frame (i.e., image) with multiple imaging systems running continuously. This capability, without human oversight of the raw data, requires tracking methods that are (1) highly scalable and (2) provide a built-in measure of confidence / diagnosis of the tracking results. Furthermore, the probabilistic tracking algorithms provided herein provide a built-in measure of confidence to the consuming application / process without human oversight. Furthermore, the current probabilistic tracking algorithms provide dynamic metrics that can be used for drug screening independent of the specific trajectory.

[0047] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims.

[0048] This patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. [Brief explanation of the drawings]

[0049] [Figure 1] A schematic diagram of the htSMT workflow is shown.

[0050] [Figure 2] FIG. 1 shows a schematic diagram of an exemplary sample acquisition system of the present disclosure.

[0051] [Figure 3] 1 illustrates a cross-sectional side view of an exemplary illumination scheme that may be used in connection with certain aspects of the present invention, specifically, thin-layer oblique illumination microscopy (HILO).

[0052] [Figure 4]FIG. 4 shows cross-sectional side views (top images) of exemplary illumination schemes, including HILO, high-inclination swept tile (HIST) microscopy, and single-objective tilted light sheet (SOLEIL) microscopy, that can be used in connection with certain aspects of the present disclosure. The bottom images show schematic camera views of each corresponding illumination scheme.

[0053] [Figure 5] 1 shows a schematic diagram of an exemplary sample handling system of the present disclosure.

[0054] [Figure 6] An exemplary system for a high-throughput single-molecule imaging platform for measuring protein movement in living cells is shown.

[0055] [Figure 7] FIG. 1 illustrates data flow through an exemplary system for a high-throughput single-molecule imaging platform that measures protein movement in living cells.

[0056] [Figure 8] 10A-10C are multiple images showing the difference between mask categories and instance / semantic masks.

[0057] [Figure 9] 1 illustrates an exemplary computer-implemented environment relevant to the subject matter described herein.

[0058] [Figure 10] FIG. 1 illustrates a sample computing device architecture for implementing various aspects described herein.

[0059] [Figure 11] A diagram showing the pipeline for scalable tracing in htSMT is shown.

[0060] [Figure 12A]We present benchmarks of various tracking algorithms and demonstrate aspects related to optical dynamic simulations. [Figure 12B] We present benchmarks of various tracking algorithms and provide a basis for comparison. [Figure 12C] We present a benchmark of various tracking algorithms and present benchmarking results in terms of recall, precision, and F1 score.

[0061] [Figure 13A] Figure 1 shows an embodiment of the high-throughput single-molecule tracking platform. The diffusion state probability distributions for three cell lines expressing histone H2B-Halo, Halo-CaaX, or Halo alone are shown. Shaded bins represent the diffusion states characteristic of each cell line. [Figure 13B] Figure 1 shows aspects of the high-throughput single-molecule tracking platform. Exemplary fields of view containing single-molecule images (top) and reference Hoechst images (bottom) of mixed H2B-Halo, Halo-CaaX, and free-Halo cell lines, as well as insets showing zoom-ins on individual cells and successive frames of individual molecules, with image intensity scaled uniformly across panels. [Figure 13C] 13B shows an embodiment of the high-throughput single-molecule tracking platform. Single-cell diffusion profiles extracted from FIG. 13B are color-coded based on their similarity to the H2B-Halo, CaaX-Halo, or free Halo dynamics reported in FIG. 13A. [Figure 13D] Figure 13 shows an embodiment of a high-throughput single-molecule tracking platform. A heatmap representation of 103,757 cell nuclei measured from a mixture of Halo-H2B, Halo-CaaX, or free Halo across 1,540 unique wells in five 384-well plates is shown. Each horizontal line represents a nucleus. Cells were clustered using k-means clustering and assigned labels based on the diffusion profiles determined in Figure 13A. [Figure 13E]Figure 1 shows an embodiment of the high-throughput single-molecule tracking platform. Ensemble state sequences of all tracks collected from a mixture of Halo-H2B, Halo-CaaX, and free Halo cells are shown.

[0062] [Figure 14A] We provide a comparison of the dynamics of a panel of steroid hormone receptors (SHRs). A diagram of SHR function is shown, demonstrating that under basal conditions, SHRs are sequestered in a complex with HSP90 and other cofactors; upon ligand binding, the receptor dissociates from the inactive complex, dimerizes, and binds to DNA. [Figure 14B] Figure 1 provides a comparison of the kinetics of a panel of steroid hormone receptors (SHRs). The distribution of the diffusion state of Halo-AR, Halo-ER, Halo-GR, and Halo-PR in U2OS cells before and after stimulation with activating ligands is shown. The area of ​​the shaded region is the fbound, and the shaded error band represents the SD. [Figure 14C] Figure 1 provides a comparison of kinetics across a panel of steroid hormone receptors (SHRs). Selectivity for individual SHRs' cognate ligands compared to other steroids as determined by fbound is shown, with error bars representing SEM. [Figure 14D] Figure 1 provides a comparison of kinetics across a panel of steroid hormone receptors (SHRs). Selectivity for individual SHRs' cognate ligands compared to other steroids, as determined by Dfree, is shown; error bars represent SEM.

[0063] [Figure 15]This figure shows that screening for bioactive compounds against the estrogen receptor (ER) demonstrates reproducibility and robustness. Reproducibility of screening results was assessed across two biological replicates. The plot shows the change in fbound relative to ER from 5,067 compounds. Magenta compounds were identified as active if the magnitude of the change in fbound exceeded ±0.05 when averaged across both replicates. Of the 30 predicted positive control compounds, 26 significantly increased fbound in both replicates (gray outline). Three compounds increased fbound but were present in only one replicate after filtering. Exemplary positive controls are illustrated, including the agonist estradiol (1) and the antagonists fulvestrant (5), 4-OHT (6), and bazedoxifene (7). Linear regressions were fitted to the magenta compounds to determine slope and correlation.

[0064] [Figure 16A] Figure 1 shows that selective ER modulators and degraders induce measurable DNA binding through SMT. Diffusion state probability distributions of ER treated with 100 nM of exemplary selective ER modulators (SERMs) and selective ER degraders (SERDs) are shown, with the shaded area representing the SD. [Figure 16B] Selective ER modulators and degraders induce measurable DNA binding through SMT. The change in fbound as a function of titration of 12 infusions of fulvestrant (5), 4-OHT (6), GDC-0810 (8), AZD9496 (9), or GDC-0927 (10) is shown as fitted curves and color-coded as in (Figure 16A). Error bars represent the SEM of three replicates. [Figure 16C] Selective ER modulators and degraders induce measurable DNA binding through SMT. The change in fbound as a function of time after addition of agonist or antagonist is shown as a single exponential fit, and compounds are color-coded as in Figure 16A. Here, estradiol (1, green) and DMSO were added for comparison, and error bars represent SEM. [Figure 16D] Selective ER modulators and degraders induce measurable DNA binding via SMT. The maximum effect of SERMs and SERDs on binding is shown. Each box represents a quartile, and the whiskers indicate the 5th to 95th percentiles of single-well measurements measured over a minimum of 4 days using eight wells per compound per day. [Figure 16E] Selective ER modulators and degraders induce measurable DNA binding through SMT. Fluorescence recovery from photobleaching (FRAP) of ER-Halo cells treated with DMSO alone or 100 nM SERM / D is shown. Curves are the mean ± SEM of 18–24 cells, color-coded as in Figure 16A, and error bands represent SEM. [Figure 16F] Selective ER modulators and degraders induce measurable DNA binding through SMT. Quantification of FRAP recovery curves measuring recovery at 2 minutes after photobleaching is shown. Here, whiskers indicate the 5th to 95th percentiles of single-cell measurements. [Figure 16G] We demonstrate that selective ER modulators and degraders induce measurable DNA binding through SMT. Track length survival curves for ER-Halo cells treated with either DMSO alone or SERM / D are shown. Track survival is plotted as the 1-CDF of the track length distribution, with faster decay indicating shorter binding times. The inset shows a quantification of the curves in Figure 16G. Each point represents the percentage of tracks lasting longer than 10 seconds for a single biological replicate consisting of 3-10 wells per condition. The dashed line indicates the median percentage of tracks lasting longer than 10 seconds for histone H2B-Halo, which represents the upper limit of measurement sensitivity.

[0065] [Figure 17A]We demonstrate that htSMT can be used to determine chemical structure-activity relationships. Examples of GDC-0927-induced ER degradation in various cell lines, as measured by immunofluorescence against ER, are shown. Cells were exposed to the compound for 24 hours before fixation, and pixel intensities in the images are scaled uniformly. [Figure 17B] Figure 1 shows that htSMT can be used to determine chemical structure-activity relationships. Correlation of potency measured by ER degradation and cell proliferation in MCF7 cells (black) and T47d cells (magenta) for compounds in the GDC-0927 structural series. [Figure 17C] We demonstrate the feasibility of using htSMT to determine chemical structure-activity relationships. The change in fbound over a 12-point dose titration of compounds 11–16 is shown, color-coded by structure. Points are the mean ± SEM of three biological replicates. [Figure 17D] Figure 1 shows that htSMT can be used to determine chemical structure-activity relationships. Correlations of potency measured by changes in fbound and cell proliferation in MCF7 cells (black) and T47d cells (magenta) are shown for compounds in the GDC-0927 structure series.

[0066] [Figure 18A] Bioactive molecules targeting ER-related pathways that affect ER dynamics are shown. The results of bioactivity screening are shown with selected inhibitors grouped by pathway and individually color-coded. [Figure 18B] Bioactive molecules targeting ER-related pathways that affect ER dynamics are shown. Figure 1 shows the change in bounds over a 12-point dose titration of three representative compounds targeting HSP90, mTOR, CDK9, and the proteasome, respectively. Individual molecules are indicated by their specific shapes. Error bars represent SEM. [Figure 18C]Bioactive molecules targeting ER-associated pathways that affect ER dynamics are shown. Changes in fbound as a function of time after compound addition are shown, comparing estradiol treatment (black) with ganetespib (blue circles) and HSP990 (blue squares). Points are 4-minute bins, and error bars represent SEM. [Figure 18D] Bioactive molecules targeting ER-associated pathways that affect ER dynamics are shown. Changes in fbound as a function of time after compound addition are shown, comparing estradiol treatment (black) with the HSP90 inhibitors ganetespib (blue circles) and HSP990 (blue squares) and the proteasome inhibitors bortezomib (red circles) and carfilzomib (red squares). Points are bins of 7.5-minute htSMT data showing the mean ± SEM. The shaded area indicates the time window used during htSMT screening. [Figure 18E] Bioactive molecules targeting ER-associated pathways that affect ER dynamics are shown. Track length survival curves for ER-Halo cells treated with either DMSO alone, estradiol stimulation, or 100 nM HSP90 or proteasome inhibition are shown. Track survival is plotted as the 1-CDF of the track length distribution. Faster decay indicates shorter binding times. The inset quantifies the number of tracks greater than 2 in length as a function of treatment condition. All conditions were normalized to the median number of tracks in DMSO. [Figure 18F] Figure 1 shows bioactive molecules targeting ER-related pathways that affect ER dynamics. Figure 2 provides a diagram summarizing pathway interactions based on htSMT results for ER, AR, and PR.

[0067] [Figure 19A] We provide a characterization of the htSMT system performance. We show the distribution of localization errors measured across multiple independent wells using histone H2B-Halo cells, with a median localization error of 39 nm. [Figure 19B]We provide a characterization of the htSMT system performance. The number of nuclei measurable within a 94 × 94 μm FOV is shown, with each box plot representing the distribution across wells in one 384-well plate.

[0068] [Figure 20A] Screening for bioactive molecules provides robust data with good assay performance. Box plots showing the number of cell nuclei measured for each compound tested, with whiskers representing the 1st and 99th percentiles. [Figure 20B] Screening for bioactive molecules generates robust data with good assay performance. Z' factor analysis of bioactivity screening is shown, where each point represents the Z' factor of a single 384-well plate measuring the difference between DMSO and 25 nM estradiol treatment; plates with very low Z' factors were excluded from further analysis. [Figure 20C] 1 shows the dose titration of estradiol for each plate in a bioactive molecule screen fitted with logistic regression to determine potency, providing that the screen for bioactive molecules generates robust data with good assay performance. [Figure 20D] Screening for bioactive molecules provides robust data with good assay performance. Figure 20C shows the EC50 values ​​extracted from each curve fit, with the shaded area representing a 3-fold range in potency. [Figure 20E] The screening of bioactive molecules provides robust data with good assay performance. The distribution of fbound changes in the control DMSO wells is shown.

[0069] [Figure 21A]Figure 1 shows that SERMs and SERDs decrease free diffusion and rapidly increase free boundary after addition. Normalized occupancy of the diffusion state (diffusion coefficients 0.2–100 μm / sec) for samples treated with 100 nM SERD or SERM compared to DMSO is shown. Histograms are normalized to integrate to 1, shaded areas represent standard deviations per bin, and curves are the average of 3–4 biological replicates with 8 replicate wells per condition. [Figure 21B] Figure 17C shows that SERMs and SERDs decrease free diffusion and rapidly increase fbound after addition. Figure 17C shows the results of fitting a single exponential relationship to the data. [Figure 21C] Figure 1 shows that SERMs and SERDs decrease free diffusion and rapidly increase fbound after addition. For stivasol and fulvestrant, the change in diffusion state distribution as a function of time after compound addition is shown. Each curve represents the average of three biological replicates, and the shaded area is the bin-wise standard deviation. [Figure 21D] 1 shows that SERMs and SERDs decrease free diffusion and rapidly increase fbound after addition. 1 shows the change in fbound of selective AR or GR antagonists from the bioactivity screening set. [Figure 21E] We show that SERMs and SERDs decrease free diffusion and rapidly increase f after addition. We provide a table of the results of the slow decay rate constant (k) from the SMT curve fit. The fit was performed on a set of three biological replicates, and combined with the f determined in Figure 17D, we can calculate an upper limit for k, inferred from the equation assuming all bound molecules have k*off = k. Cells marked with an asterisk are cells where k could not be reliably distinguished from photobleaching.

[0070] [Figure 22A]We present GDC-0927 structural variants characterized by ER degradation or cell proliferation assays. Western blots of ER-expressing breast cancer cell lines MCF7 and T47d were compared with ER expression in ER-null lines SK-BR-3 and U2OS. Samples were treated with fulvestrant for 24 hours before lysis. Fulvestrant treatment induces ER degradation, even when fused to HaloTag. [Figure 22B] 1 provides GDC-0927 structural variants characterized by ER degradation or cell proliferation assays. 1 shows exemplary compound dose titrations showing the change in mean nuclear intensity as a function of compound concentration. [Figure 22C] GDC-0927 structural variants characterized by ER degradation or cell proliferation assays are provided. An example of the effect of GDC-0927 on MCF7 breast cancer cell proliferation is shown, compared to staurosporine as a positive control. [Figure 22D] GDC-0927 structural variants are provided that are characterized by ER degradation or cell proliferation assays. Examples of compound dose titration are shown, measuring cell proliferation in cells treated with analogs of GDC-0927 and normalizing to DMSO-treated cells.

[0071] [Figure 23A] Further exploration of bioactivity screening data is provided, including non-limiting exemplary cutoffs for active molecules. Examples of compound effects on ER bound from dose titration experiments are shown, where 92 compounds from the primary screen are ranked based on effect size. Compounds colored in black repeatedly showed dose-dependent changes, while compounds colored in magenta were inactive in dose titration. [Figure 23B] Further exploration of bioactivity screening data is provided, including non-limiting exemplary cutoffs for active molecules. Exemplary dose titrations of compounds that result in increasing overall changes in ER fbound are shown. [Figure 23C]Further exploration of bioactivity screening data, including non-limiting exemplary cutoffs for active molecules, is provided. Figure 16 shows two replicates of a bioactivity screen similar to Figure 15, with active compounds color-coded based on their structural scaffold. [Figure 23D] Further exploration of the bioactivity screening data, including non-limiting exemplary cutoffs for active molecules, is provided. Quantification of the number of compounds associated with any cluster is shown, with singletons (cluster 21) representing the majority of active compounds.

[0072] [Figure 24A] We demonstrate that several inhibitors of pathways that regulate ER dynamics are ER-specific. Figure 1 shows changes in ER bounds following treatment with inhibitors of HSP90, proteasome, mTOR, and CDK from an initial bioactivity screen. CDK inhibitors are further classified as CDK4 / 6 inhibitors, CDK9 inhibitors, or inhibitors without strong selectivity for a particular family (pan-CDK). Lines represent the median values ​​for each target. [Figure 24B] We demonstrate that some pathway inhibitors that regulate ER dynamics are ER-specific. The ER bound changes for 97 bioactive molecules are illustrated, colored by their pathway annotation. Compounds are ranked based on their effect on the ER. Error bars are SEM of three biological replicates. [Figure 24C] This indicates that some inhibitors of pathways that regulate ER dynamics are ER-specific. Figure 24B shows the changes in AR bound for the same molecules as in Figure 24B, with compounds ranked according to their effect on the ER. Error bars are SEM of three biological replicates. [Figure 24D] This shows that some inhibitors of pathways that regulate ER dynamics are ER-specific. Figure 24B shows the changes in PR bound for the same molecules as in Figure 24B, where the compounds are ranked based on their effect on the ER. Error bars are SEM of three biological replicates.

[0073] [Figure 25] 1 shows a dose titration plot of ER (S104A / S106A / S118A) with mTOR and CDK9 compounds. Each point represents the mean and SEM of three biological replicates.

[0074] [Figure 26] Shown are fbounds measured for the AR in the presence of 25 nM agonist, 10 μM strong antagonist, or a combination of 25 nM agonist and 10 μM antagonist.

[0075] [Figure 27] Figure 1 shows the change in target A movement in response to dose titration of a known target A antagonist, with a series of shapes representing different compounds and error bars representing SEM.

[0076] [Figure 28] Two representative examples of receptor tyrosine kinases (Target B and Target C) are shown whose kinetics change in response to dose titration of known ATP-competitive or allosteric inhibitors. Each series of shapes represents a different compound normalized to DMSO, and error bars represent SEM.

[0077] [Figure 29] Changes in Q3 jump length as a function of time after compound addition are shown for additional representative targets. Treatment of the protein with an on-target inhibitor increases or decreases protein motion within minutes of compound addition, for target B or target A, respectively. As in the case of DNA damage induction, treatment of the helicase with either a pathway antagonist or an off-target modulator induces changes in protein motion that take several hours or longer to reach maximum effect.

[0078] [Figure 30] The cumulative number of tracks as a function of imaging time for each individual cell type labeled with 10 pM JF549 is shown. Shaded error bars represent 1 standard deviation.

[0079] [Figure 31] The mRNA transcription levels (log(FPKM)) of AR, ESR1, PR, and NR3C1 are shown for the modified U2OS cell line compared to the parental U2OS cell line and three reference breast cancer cell lines.

[0080] [Figure 32] The reference gene set induced 24 hours after stimulation with 25 nM estradiol is shown. The top five induced gene sets for increased Halo-ER ectopic expression were not significantly induced in the parental U2OS line.

[0081] [Figure 33] A bar plot showing the -log(q value) from the top 50 most significantly induced gene sets after estradiol stimulation is shown. Gene sets characteristic of ESR1 or estrogen response are annotated in dark grey.

[0082] [Figure 34] The cumulative number of trajectories as a function of imaging time for DMSO-treated cells labeled with 20 pM JF549 is shown. Shaded error bars represent 1 standard deviation.

[0083] [Figure 35] Changes in fbound for 30 known ER-interacting molecules are shown and ordered by effect size, circled in Figure 15. Error bars are SEM.

[0084] [Figure 36] Figure 1 shows that antagonists of ER (A), PR (B), AR (C), and GR (D) have different effects on the dynamics of their target proteins. Changes in fbound following the addition of 1 μM antagonist in the absence or presence of the cognate agonist. The dashed line indicates the fbound of the solvent-treated control. DETAILED DESCRIPTION OF THE INVENTION

[0085] The subject matter of this disclosure relates to a major industrial-scale high-throughput SMT (htSMT) technology, systems incorporating such htSMT technology, hardware and software developments related to such htSMT technology, and methods using such htSMT technology. For example, the htSMT technology described herein is capable of measuring protein movement in over 1,000,000 cells per day. Furthermore, using the estrogen receptor (ER) as a proof-of-concept system, the htSMT technology described herein demonstrates specific, robust, and reproducible results. The htSMT technology described herein can be used for a variety of applications, including, but not limited to, drug discovery activities such as screening compound libraries and elucidating structure-activity relationships (SAR). Importantly, the htSMT technology described herein can be used to characterize the contribution of both known and novel pathways to larger molecular assemblies containing targets, such as protein signaling interaction networks.

[0086] 1 , aspects of the present subject matter can be implemented using an htSMT workflow, which can include various stages, such as (i) sample preparation, including reagent handling, (ii) image acquisition using imaging of the sample to generate a series of images and / or videos, (iii) image analysis, for example, using various analytics, single emitter detection and sub-pixel localization (i.e., "super-resolution imaging"), tracking, computer vision, and machine learning algorithms to process these images and videos, (iv) storage of information extracted from or characterizing or constituting the images and videos (i.e., features, raw images, modified images, etc.), and (v) providing insights using the stored information, including biological interpretations (which can additionally or alternatively be provided using various analytics, tracking, computer vision, and machine learning algorithms), as described in more detail below.

[0087] The subject matter of the present disclosure will be described with reference to the figures, wherein reference numerals are used to denote like or equivalent elements throughout. The figures are not drawn to scale and are provided solely to illustrate aspects disclosed herein. Certain disclosed aspects are described below with reference to illustrative example hardware, software, and applications. It should be understood that numerous specific details, relationships, and methods are set forth to provide a more thorough understanding of the subject matter disclosed herein. For clarity of disclosure, and not for purposes of limitation, the detailed description is divided into the following subsections. 1.Definition 2. htSMT hardware 3. htSMT software 4. Specific htSMT Applications 5. Exemplary Embodiments 6. Working Example

[0088] 1.Definition Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. In the case of conflict, the present document, including definitions, will control. Preferred methods and materials are described below; however, methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the subject matter of this disclosure. All publications, patent applications, patents, and other references mentioned herein are incorporated herein by reference in their entirety. The materials, methods, and examples disclosed herein are illustrative only and are not intended to be limiting.

[0089] The terms "comprise," "include," "having," "has," "can," "contain," and variations thereof, as used herein, are intended to be open-ended transitional phrases, terms, or phrases that do not exclude the possibility of additional acts or structures. The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. The present disclosure also contemplates other embodiments that "comprising," "consisting of," and "consisting essentially of" embodiments or elements present herein, whether or not explicitly stated.

[0090] In reciting numerical ranges herein, each intervening number in the range is expressly contemplated with the same precision. For example, in the range 6 to 9, the numbers 7 and 8 are contemplated in addition to 6 and 9, and in the range 6.0 to 7.0, the numbers 6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, and 7.0 are expressly contemplated.

[0091] As used herein, the term "about" or "approximately" means within an acceptable error range for a particular value as determined by those skilled in the art, which depends in part on the method of measuring or determining the value, i.e., the limitations of the measurement system. For example, "about" can mean within 3 or more standard deviations, according to the practice in the art. Alternatively, "about" can mean a range of up to 20%, preferably up to 10%, more preferably up to 5%, and even more preferably up to 1% of a given value. Alternatively, particularly with respect to biological systems or processes, this term can mean within an order of magnitude, preferably within 5 times, more preferably within 2 times of a value.

[0092] As used herein, the term "trajectory" refers to a set of spatial coordinates corresponding to the observed positions of fluorescent proteins linked in time. In certain cases, multiple trajectories can be constructed algorithmically by linking multiple fluorescent proteins whose positions have been determined at successive time points. In certain cases, multiple trajectories can be constructed conservatively by linking only spots within a fixed search radius when other links are not valid. In certain cases, multiple trajectories can be constructed probabilistically.

[0093] As defined herein, protein motion refers to changes in the positions of multiple fluorescent proteins. In certain cases, protein motion can be quantified by analyzing changes in spatial coordinates at successive time points. Motion characterized in this way can also include, but is not limited to, measurements of jump length distribution. Given a set of protein displacements between one time point and a subsequent time point, a histogram of the probability of each displacement length ("jump length") can be constructed. Quantiles of this distribution can be used to describe protein motion. In certain cases, the quantile used is the median of the jump length distribution. In certain cases, the quantile used is the third quartile of the jump length distribution. In certain cases, protein motion can be quantified by analysis of trajectories. Motion characterized in this way can include, but is not limited to, measurements of mean square displacement, defined as the mean value of the squares of all displacements within a trajectory, averaged over multiple trajectories. Motion characterized in this way can also include, but is not limited to, measurements of trajectory length or measurements of trajectory length distribution. Such characterized motion may include, but is not limited to, measurements of the average radius of gyration, defined by the root-mean-square distance of all coordinates on a trajectory from the center of mass of the set of points included in the trajectory, averaged over multiple trajectories. Such characterized motion may also include, but is not limited to, measurements of the average bond angle, defined by the angle formed from three consecutive spatial coordinates averaged over multiple trajectories. Such characterized motion may also include, but is not limited to, measurements of the maximum likelihood diffusion coefficient estimate, defined as the maximum likelihood diffusion coefficient estimate for multiple trajectories under a single-state diffusion model with a constant localization error. In certain cases, protein motion may be measured through analysis of the products of a link generation algorithm. Such characterized motion may include, but is not limited to, the mean posterior diffusion coefficient, the average of the posterior probability distribution of coefficients from a probabilistic linkage algorithm. Such characterized motion may include, but is not limited to, the geometric mean posterior diffusion coefficient, the average of the log-scaled posterior probability distribution of coefficients from a probabilistic linkage algorithm.In certain cases, protein motion can be measured through model-dependent analysis of multiple trajectories. Motion characterized in this way is the fraction of immobile molecules ("f") defined by two-state model fitting. bound ") may include, but is not limited to:

[0094] As used herein, the term "motion" encompasses not only changes in the direction in which a target moves, but also both increases and decreases in the speed of movement. Thus, tracking motion may, in certain cases, include determining that a target is not moving, e.g., determining that the target is in a statically constrained state or an essentially statically constrained state. Motion can be characterized in various ways, including, but not limited to, quantifying (a) the median of the jump length distribution (where the jump length corresponds to the observed distance traveled by the target's fluorescent protein in successive frames), (b) the third quartile of the jump length distribution, (c) the median radius of gyration, (d) the mean posterior diffusion coefficient, (e) the geometric mean posterior diffusion coefficient, (f) the mean squared displacement, (g) the median bond angle, (h) the maximum likelihood estimator of the diffusion coefficient, (i) the trajectory length, and / or (j) the inferred state occupancy.

[0095] As used herein, detected movement includes, but is not limited to, any change in movement, which may occur in response to any environmental or other factor. For example, but not limited to, movement, or lack thereof, may be caused by (A) the addition of a compound, (B) a change in temperature, (C) a change in oxygen concentration, e.g., the introduction of hypoxia, (D) mechanical stress, (E) a change in pH, and / or (F) a change in light exposure (e.g., an increase or decrease in intensity).

[0096] As used herein, the term "fluorescent protein" refers to any protein that emits a fluorescent signal. In certain cases, the fluorescent emission occurs in response to irradiation with light of a specific wavelength. An example of a naturally occurring fluorescent protein is green fluorescent protein (GFP). However, in certain cases, a protein of interest can be adapted to emit a fluorescent signal through the introduction of an encoded fluorescent tag. That is, a protein sequence is fused to the protein of interest to make it fluorescent. In certain cases, a protein of interest can be adapted to emit a fluorescent signal through the binding of a fluorescent ligand. Non-limiting examples of such encoded fluorescent tags include Halo tag, SNAP tag, CLIP tag, TMP tag, and SunTag. Additionally or alternatively, a protein of interest can be adapted to emit a fluorescent signal by binding to a fluorescent dye molecule, such as an amine-reactive dye or a sulfhydryl-reactive dye.

[0097] As used herein, the term "compound" refers to any chemically defined entity. In certain cases, a compound may be a molecule less than 1000 Da, i.e., a "small molecule." In certain cases, a compound may be a macromolecule, such as a nucleic acid. In certain cases, a nucleic acid may have a defined sequence. In certain cases, a nucleic acid includes (A) ribonucleic acid (RNA) (e.g., including modified RNA), (B) deoxyribonucleic acid (DNA) (e.g., including modified DNA), and (C) a combination of (A) and (B). In certain cases, a nucleic acid is a single- or double-stranded small interfering nucleic acid (e.g., double-stranded siRNA), an antisense oligonucleotide, a ribozyme, a microRNA, or an aptamer. In certain cases, a compound may be a protein. For example, and not by way of limitation, protein compounds of the present disclosure include signaling proteins, such as protein hormones, cytokines, kinases, phosphatases, and other enzymes and transcription factors, as well as antibodies, contractile proteins, structural proteins, storage proteins, and transport proteins.

[0098] In certain cases, a compound can refer to a mixture of molecules, for example, a mixture of defined composition.

[0099] Throughout the drawings and specification, specific numbers are associated with specific compounds. See, e.g., Figures 14B, 15, 16A-16C, 17C, and Example 1. Specifically, the compounds and their respective numbers are estradiol (1), 2-hydroxytestosterone (2), progesterone (3), dexamethasone (4), fulvestrant (5), 4-OHT (6), bazedoxifene (7), GDC-0810 (8), AZD9496 (9), and GDC-0927 (10). Additionally, Figure 17C includes the GDC-0927 structure series, where specific modifications to GDC-0927 are indicated and numbered (11) through (16).

[0100] 2. htSMT hardware 2.1.Image Acquisition System

[0010] Referring to Figure 1, aspects of the present subject matter can be implemented using an htSMT workflow, where such a workflow incorporates image acquisition using imaging of a sample to generate an image sequence and / or video. For example, without limitation, Figure 2 shows a schematic diagram of an exemplary image acquisition system of the present disclosure. The exemplary image acquisition system (2-001) comprises a light source and a single mode fiber (SMF) (2-002) configured to emit light (2-003), the light (2-003) being relayed by one or more optical elements in an optical relay (2-004), the optical relay being configured to shape the light emitted from the light source to form a shaped beam (2-005), the system further comprising one or more optical elements (2-006), such as a dichroic mirror, that transmit the shaped beam through a water immersion microscope (WIM) (2-006). The objective (2-008) is configured to direct the sample surface (2-010) so that the sample surface (2-010) is illuminated by the oblique beam (2-009), resulting in emission from the sample (2-012), e.g., fluorescent emission, which is focused onto the objective, and this emission is focused by the water immersion objective (2-008) and one or more optical elements (2-013), e.g., a tube lens, and passed through an absorption filter wheel (2-014) to an image acquisition system (2-015), e.g., a detector device.

[0101] 2.1.1.Light source Referring to the exemplary image acquisition system of FIG. 2, the system includes a light source (2-002) configured to emit light (2-003). The light source (2-002), in certain embodiments of the image acquisition systems disclosed herein, can be configured to emit light at a single wavelength. In certain embodiments of the image acquisition systems disclosed herein, the light source (2-002) can be configured to emit light at two, three, four, five, or more distinct wavelengths. In certain embodiments, the wavelength(s) of light emitted by the light source are predetermined. For example, and not by way of limitation, the wavelength(s) can be predetermined such that, when irradiated onto a sample, e.g., a sample containing a fluorescent protein, the emitted light induces fluorescence. In certain instances, the wavelength(s) used in connection with the methods described herein will fall within the range of 400 nm to 650 nm. In certain instances, the light source (2-002) emits light having a wavelength of 400 nm to 408 nm, 550 nm to 565 nm, or 638 nm to 650 nm. In certain non-limiting embodiments, the light source (2-002) is configured to include three lasers with nominal center wavelengths of 405 nm, 560 nm, and 640 nm, which can vary within the absorption band of the fluorophore used. In certain instances, the 405 nm wavelength is used to excite Hoechst dye. In certain instances, the dye (e.g., JF) attached to the HaloTag is used to excite the dye. 549 A wavelength of 560 nm is used to excite .

[0102] In certain non-limiting embodiments, the light source (2-002) is used to catalyze a photochemical reaction. For example, but not by way of limitation, the wavelength(s) and irradiation intensity can be such that cleavage of a chemical bond occurs. As a further example, but not by way of limitation, the wavelength(s) and irradiation intensity can induce the adoption of a non-radiative dark state (i.e., "photobleaching molecules"). As a further example, but not by way of limitation, the wavelength(s) and irradiation intensity can induce radiative or non-radiative energy transfer between fluorophores within the sample.

[0103] In certain implementations of the image acquisition system described herein, the light source (2-002) can be configured to deliver a predetermined amount of power to the back focal plane of the objective lens (2-007). For example, and without limitation, the light source (2-002) can deliver more than 10 mW of power at a particular wavelength, such as 405 nm, and / or more than 150 mW of power at another wavelength, such as 640 nm. Additionally or alternatively, if the light source (2-002) includes three lasers emitting at wavelengths of 405 nm, 560 nm, and 640 nm, respectively, the light source (2-002) can be configured to deliver a predetermined amount of power to the back focal plane of the objective lens (2-007). For example, and without limitation, 405 nm can be configured to deliver less than 10 mW, 560 nm can be configured to deliver more than 150 mW, and 640 nm can be configured to deliver more than 50 mW.

[0104] In certain embodiments of the image acquisition systems described herein, the light source (2-002) is configured to emit pulsed light. For example, and not by way of limitation, the light source (2-002) may be configured to emit strobe pulsed light. In one particular, non-limiting embodiment, the light source (2-002) emits a 2 msec strobe pulsed light. Additionally or alternatively, the light may be pulsed in synchronization with the start of frame acquisition, as described in detail below.

[0105] The emission of light (2-003) by the light source (2-002) and the directing of that light to the optical relay (2-004) may be facilitated using a single mode fiber in certain embodiments of the image acquisition systems disclosed herein. Additionally or alternatively, multimode fibers with predetermined core geometries for sample illumination may be used.

[0106] In certain embodiments of the image acquisition systems described herein, for example, with respect to systems configured for high-throughput sample analysis, the light source (2-002) can be configured to exhibit low drift in output. In certain embodiments, such low-drift configurations increase the consistency of sample processing and facilitate high-throughput analysis. For example, and not by way of limitation, such low-drift output configurations maintain output within about 0% to about 15% variation, about 0% to about 10% variation, about 10% variation, about 9% variation, about 8% variation, about 7% variation, about 6% variation, about 5% variation, about 4% variation, about 3% variation, about 2% variation, or about 1% variation.

[0107] In certain instances, such a low-drift output configuration maintains output power within about 0% to about 15% variation, about 0% to about 10% variation, about 10% variation, about 9% variation, about 8% variation, about 7% variation, about 6% variation, about 5% variation, about 4% variation, about 3% variation, about 2% variation, or about 1% variation across ambient (room) temperature variations of, for example, 17°C + / - 5°C. In certain instances, this is achieved by using a temperature sensor and / or a closed-loop heater to stabilize the temperature of the internal light source (e.g., laser engine) and thereby reduce output power drift. For example, but not by way of limitation, the light source may be thermally isolated from ambient temperature variations using an insulated enclosure design. Additionally or alternatively, closed-loop heaters can be strategically placed in specific locations within the system, such as at the fiber coupler, to reduce output power drift. Additionally or alternatively, a water jacket and / or cooling device can be used to reduce heat buildup from the laser head. Additionally, these thermal controls, used individually or in combination, reduce the warm-up time to reach a steady state of operation and maintain a more stable internal operating temperature as the laser is powered off and on.

[0108] 2.1.2. Optical Elements and Sample Illumination Referring to the exemplary image acquisition system of FIG. 2A, the system includes a light source (2-002) configured to emit light (2-003), which is relayed by one or more optical elements in an optical relay (2-004), which is configured to shape the light emitted from the light source to form a shaped beam (2-005).

[0109] While FIG. 2 illustrates an exemplary HILO implementation for use in the htSMT workflow described herein, the htSMT workflow described herein can incorporate a variety of illumination strategies. For example, and not by way of limitation, the htSMT workflow described herein can be implemented using HILO, total internal reflection fluorescence (TIRF), HIST, or SOLEIL microscope illumination strategies. Based on the htSMT workflow described herein, those skilled in the art will understand advantageous ways to adapt TIRF, HIST, or SOLEIL illumination strategies for use in the present methods. For example, the optical elements of a particular optical relay (2-004) can be selected and configured to generate an appropriately shaped beam (2-005) and provide appropriate transformation of that beam, for example, if a HIST illumination strategy is employed. Additionally or alternatively, those skilled in the art will understand how to configure the optical elements necessary to achieve a TIRF illumination strategy based on the htSMT workflow described herein. For example, optical elements can be used to sharply tilt the beam until it reaches its critical angle, thereby causing an evanescent wave to propagate through the cover glass and illuminate a sample proximate to the cover glass.

[0110] In certain non-limiting embodiments of the optical relay (2-004) of the image acquisition system of the present disclosure, the optical relay (2-004) comprises one or more lenses. For example, but not by way of limitation, the selection and orientation of the lenses in the optical relay (2-004) are configured to appropriately shape the light beam directed to the sample. In certain non-limiting embodiments, the optical relay (2-004) comprises a lens having a predetermined focal length (e.g., 80 mm) to collimate the emitted light (2-003) from the light source (2-002). Additionally or alternatively, the optical relay (2-004) includes a lens or series of lenses, such as a telescope system, for shaping the light beam. The particular focal length(s) of the lens or series of lenses are predetermined to generate a light beam of an appropriate shape.

[0111] In embodiments of the htSMT workflow described herein, the image acquisition system is configured to incorporate a HIST microscope-based illumination system, and the optical relay can be configured to include a telescope with two cylindrical lenses (e.g., f=400 / 250 mm and f=50 mm) to generate a tile beam compressed by 8x or 5x; in certain embodiments, this tile beam is relayed by another telescope system (e.g., f=60 mm and f=150 mm) before passing through an additional lens (e.g., f=400 mm). In such embodiments of the HIST microscope-based illumination system, the optical relay (2-004) can include one or more optical elements or assemblies configured to translate the light beam relative to the imaging plane of the analyzed sample, e.g., in a direction perpendicular to the longer dimension of the light beam. For example, but not by way of limitation, the optical element or assembly configured to translate the light beam relative to the imaging plane of the analyzed sample can include a galvo mirror. Additionally or alternatively, such optical elements or assemblies configured to translate the light beam relative to the imaging plane of the sample being analyzed may comprise computer-controlled motors.

[0112] Referring to the exemplary image acquisition system of Figure 2, the system includes an optical relay (2-004) configured to shape light emitted from a light source to form a shaped beam (2-005), which is directed by an optical element (2-006), such as a dichroic mirror, and directed to a water immersion objective (2-008), whereby the sample plane (2-010) is illuminated by an oblique beam (2-009). The inset shows the illumination view (2-011) for the X and Y axes of the sample plane, with a gradual decrease in intensity following a Gaussian distribution from a peak intensity core (2-011A) to a lower intensity outer edge (2-011B).

[0113] In certain non-limiting embodiments of the image acquisition system of the present disclosure, the water immersion objective (2-008) directs the inclined beam (2-009) onto the sample plane (2-010) to be analyzed. In certain non-limiting embodiments of the image acquisition system of the present disclosure, the objective (2-008) is a water immersion objective. The use of a water immersion objective facilitates high-throughput sample analysis by eliminating the oil present in association with the use of oil immersion objectives, thereby enabling consistent sample handling and imaging. For example, and without limitation, the objective can be a 60x 1.27NA water immersion objective (Nikon). In certain embodiments of the workflow described herein, the water immersion objective (2-008) is heated by a heating element. For example, such a heating element maintains the water immersion objective (2-008) at a temperature sufficient to avoid inducing temperature changes to the samples contained in the sample plate (2-021).

[0114] 2.1.3. Image Acquisition In certain non-limiting embodiments of the image acquisition system of the present disclosure, the water immersion objective (2-008) is also used to focus the fluorescence emitted by the sample in response to illumination provided by the tilted beam (2-009). However, in certain cases, a second objective is used to focus the fluorescence emitted by the sample in response to illumination provided by the tilted beam (2-009). In certain non-limiting embodiments, the fluorescence emission (2-012) focused onto the objective passes through an absorption filter wheel (2-014), e.g., a bandpass absorption filter matched to the spectrum of the fluorophore under observation and mounted on a high-speed filter wheel (Finger Lakes Instruments), and is collected by a detector device (2-015). In certain non-limiting embodiments, the fluorescence emission focused onto the objective is directed to an optical relay before collection by the detector device (2-015). For example, and not by way of limitation, such an optical relay can include one or more lenses and one or more additional optical elements, such as elements configured to reject additional scattered light before collection by the detector device (2-015). In certain non-limiting embodiments, the fluorescent emission focused onto the objective lens is directed through another dichroic mirror to split the emission across multiple regions of the detector (2-015). Here, the detector device can be a CMOS camera, such as a back-illuminated CMOS camera (Prime95b, Teledyne).

[0115] In certain non-limiting embodiments in which the image acquisition system is configured to incorporate a HIST or SOLEIL microscope-based illumination system, the detector device can be configured to synchronize detection with translation of the tilted beam (2-009) across the sample. Such synchronization is shown schematically in the images associated with the HIST and SOLEIL embodiments at the bottom of FIG. 4, where "active pixels" correspond to aspects of the detector device that actively collect in synchronization with the translation of the tilted beam (2-009). For example, but not by way of limitation, the detector device can be a CMOS camera, such as a back-illuminated CMOS camera (Hamamatsu Fusion BT).

[0116] In certain embodiments of the image acquisition system of the present disclosure, the CMOS camera can be operated to collect a series of SMT frames for each field of view. For example, and without limitation, 1 to 100,000 SMT frames, 1 to 50,000 SMT frames, 1 to 20,000 SMT frames, 1 to 10,000 SMT frames, 1 to 1,000 SMT frames, 1 to 500 SMT frames, 5 to 250 SMT frames, 10 to 200 SMT frames, 100 to 200 SMT frames, or 200 SMT frames can be collected per field of view. In certain embodiments, the CMOS camera can be configured to operate at a frame rate of 0.5 to 1000 Hz. In certain embodiments, the CMOS camera can be configured to operate at a frame rate of approximately 100 Hz.

[0117] In certain non-limiting embodiments of the image acquisition system of the present disclosure, the detector device is configured to transmit a signal with each frame to trigger other components of the imaging system. For example, but not by way of limitation, the detector device can trigger illumination from the light source (2-002) to collect fluorescent emission associated with a strobe laser pulse. For example, but not by way of limitation, such fluorescent emission collection is associated with a 10-100 millisecond frame and a 2 millisecond strobe laser pulse. In certain embodiments, the fluorescent emission collection is associated with a strobe laser pulse of about 1 to about 4 milliseconds, e.g., about 1 to about 3 milliseconds, or about 2 to about 3 milliseconds, and the duration of the strobe laser pulse can be selected based on the frame rate used (e.g., 10-100 millisecond frames).

[0118] In certain embodiments, the imaging acquisition system can be configured to detect a predetermined field of view. In certain embodiments, the detection field of view can have a size of a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm). For example, and not by way of limitation, the detection field of view can have a size of a first dimension (about 94 μm) by a second dimension (about 94 μm).

[0119] In certain embodiments, a detector device can be used to collect fluorescent emissions at multiple wavelengths. For example, but not by way of limitation, fluorescent emissions of additional fluorophores can be collected at the same frame rate or at different frame rates for the same field of view to provide downstream registration of SMT tracks to other cellular components, such as nuclei. Additional channels of the detector device can be used as needed to expand the number of fluorescent emissions simultaneously captured for the same field of view to provide downstream registration of SMT tracks to other cellular components, such as nuclei.

[0120] 2.2. Sample handling Referring to FIG. 1 , embodiments of the present subject matter can be implemented using an htSMT workflow incorporating a system for sample preparation, including reagent processing. For example, and not by way of limitation, FIG. 5 provides a schematic diagram of a sample plate (2-021) with multiple wells, including, e.g., well (2-016), in which sample preparation and analysis can be performed. FIG. 5 also provides a schematic diagram of the components of a sample, e.g., cells (2-018) and fluorescent proteins (2-017) within the cells. However, as noted herein, FIG. 5 is not intended to convey scale; for example, each sample present in well (2-016) may contain thousands of cells, each cell containing numerous fluorescent proteins. FIG. 5 also schematically illustrates the ability of the sample handling system of the present disclosure to add additional reagents (2-019) to samples in the sample plate (2-021). The addition of such reagents can be handled by robotic manipulation, including but not limited to, translation of a robotic fluid handling system relative to the individual wells (2-016) of the sample plate (2-021), translation of the sample plate (2-021) itself, or a combination of both.

[0121] In certain embodiments of the image acquisition system, the sample plate (2-021) may be maintained in a temperature-controlled environment via the environmental control area (2-020). For example, but not limited to, the sample may be maintained at 22-50°C. In certain embodiments of the image acquisition system, the sample plate (2-021) may be maintained in a humidity-controlled environment via the environmental control area (2-020). For example, but not limited to, the sample may be maintained at 20%-95% humidity. In certain embodiments of the image acquisition system, the sample plate (2-021) may be maintained in a defined gas environment via the environmental control area (2-020). For example, but not limited to, the sample may be maintained at 5% CO2.

[0122] 2.2.1. Cell Lines and Cell Culture Referring to FIG. 5, a particular advantage of the htSMT system described herein is its ability to assay live cells (2-018), facilitating tracking of protein activity, mobility, and diffusion behavior within a dense, live-cell environment. As shown in FIG. 5, the htSMT system of the present disclosure can be used to track fluorescently labeled proteins in a sample containing multiple cells. Exemplary cells (e.g., cell lines) for use in conjunction with the htSMT system described herein are considered if the sample (e.g., containing such cells) can be focused by the water-immersion objective (2-008) for a sufficient period of time so that the fluorescent emission of the fluorophore can be directed toward the detector device (2-015). For example, but not by way of limitation, cells can be directly attached to a coverslip. As a further example, but not by way of limitation, the coverslip can be treated with an extracellular matrix material (e.g., fibronectin, collagen, poly-D-lysine, laminin, Matrigel, vitronectin, etc.) before inducing cells to attach to the coverslip. As an additional non-limiting example, cells can be induced to adhere to a coverslip after treating the coverslip with plasma.

[0123] Exemplary cells, e.g., cell lines, can be selected to minimize non-fluorophore emissions reaching the detector. In certain embodiments, cells for use in the present disclosure can be mammalian cells, bacterial cells, or fungal cells. In certain embodiments, the cells are mammalian cells. In certain embodiments, the cells can be obtained from preserved tissue, e.g., fixed tissue, frozen tissue, e.g., frozen tissue samples, or fresh tissue, e.g., fresh tissue samples. In certain embodiments, the cells and / or samples comprising cells can be obtained from a subject. In certain embodiments, the cells can be obtained from a tissue malignancy or tumor, e.g., the cells can be present within a tumor sample (e.g., a portion of a tumor). In certain embodiments, the cells can be obtained from a cell line. For example, but not by way of limitation, certain cell lines that can be used in connection with the htSMT system described herein include U2OS cells (ATCC catalog number HTB-96), MCF7 cells (ATCC catalog number HTB-22), T47d cells (ATCC catalog number HTB-133) and SK-BR-3 cells (ATCC catalog number HTB-30).In certain embodiments, cells can be present in three-dimensional structures such as organoids or spheroids.In certain embodiments, cells can be present in organoids.

[0124] In certain embodiments of the htSMT system of the present disclosure, the cells used are cultured as needed to provide sufficient cell numbers to achieve the desired high-throughput analysis. For example, but not limited to, cells such as U2OS cells (ATCC Catalog No. HTB-96), MCF7 cells (ATCC Catalog No. HTB-22), T47d cells (ATCC Catalog No. HTB-133), and SK-BR-3 cells (ATCC Catalog No. HTB-30) can be grown in DMEM (Cat. No. 1056601, Gibco DMEM, High Glucose, GlutaMAX Supplement, Thermofisher) supplemented with 10% fetal bovine serum (Cat. No. 16000044, Thermofisher) and 1% penicillin-strep (Cat. No. 15140122, Thermofisher), maintained in a humidified 37°C incubator with 5% CO2, and subcultured approximately every 2-3 days. Additional culture strategies that may be suitable for use with the cell lines outlined herein will be known to those of skill in the relevant art.

[0125] In certain embodiments of the htSMT system of the present disclosure, cells contain one or more fluorescent proteins. The choice of the particular protein(s) and how they fluoresce, for example, whether they are labeled via conjugation to a dye or via the inclusion of an encoded fluorescent tag, can vary depending on the specifics of a particular study. For example, but not by way of limitation, one approach for labeling proteins used in connection with the htSMT system described herein is the HaloTag fusion strategy. For example, but not by way of limitation, one approach for labeling is fluorescent proteins. For example, but not by way of limitation, one approach for labeling is photoconvertible fluorescent proteins. For example, but not by way of limitation, one approach for labeling is photoactivatable fluorescent proteins. For example, but not by way of limitation, one approach for labeling proteins is SNAPtag fusion. For example, but not by way of limitation, one approach for labeling proteins is CLIPtag fusion. For example, but not by way of limitation, one approach for labeling proteins is using a fluorophore-ligase system. For example, but not limited to, one approach to labeling proteins is to use FlAsH or ReAsH tetracysteine ​​motifs. For example, but not limited to, one approach to labeling proteins is by strain-promoted alkyne-azide cycloaddition of fluorophores. For example, but not limited to, one approach to labeling proteins is by inducing cellular uptake of separately produced fluorescent proteins. In certain embodiments of the htSMT system of the present disclosure, cells contain one or more fluorescent glycoproteins. In certain embodiments, one approach to labeling proteins uses a gene editing system, for example, a CRISPR-based editing system.For example, but not by way of limitation, a nucleic acid encoding a fluorescent protein (e.g., a fluorescent tag such as HaloTag) can be inserted into a gene encoding the protein to be labeled, or upstream or downstream of the gene, to produce a protein fluorescently labeled with HaloTag (e.g., at its C-terminus or N-terminus).

[0126] While those skilled in the art can implement the HaloTag fusion approach in a variety of ways, one exemplary approach is to transfect a mammalian expression vector containing a fusion gene (i.e., a protein of interest fused in frame with the HaloTag sequence) under the control of a weak L30 promoter and containing a neomycin resistance marker into a cell line of interest (e.g., U2OS cells). In certain embodiments, such transfection can be achieved when cells are at 70% confluence using FuGENE6 (Cat. No. E2691, Promega). In certain embodiments, transfected cells can then be selected using an appropriate selection agent, e.g., G418 (Cat. No. 10131027, ThermoFisher), at an appropriate concentration, e.g., 500 μg / mL. In certain embodiments, cells can then be clonally isolated. Clones expressing the desired fusion gene can be initially transfected using 100 nM JF 549 -HTL (Cat. No. GA1110, Promega) and 50 nM Hoechst 33342 staining, JF 549The distribution of signals can be determined by identifying expected clones. An alternative exemplary approach is to transfect cells with a ribonucleoprotein (RNP) complex containing a target protein and an sgRNA targeting a genomic sequence encoding the N- or C-terminal region of the Cas9 protein, combined with one or more linear dsDNA donors. In certain embodiments, each donor consists of 200-300 bp of homology arms specific for each target, a codon-optimized HaloTag sequence, and a TEV linker (ENLYFQG) between the target and the HaloTag. In certain embodiments, 3-6 clones can be subsequently tested for response to control compounds using SMT conditions, and the most homogeneous clones can then be expanded for further testing.

[0127] Although the htSMT workflow of the present application is generally described with respect to an embodiment tracking the effects of compounds on a target fluorescent protein, the htSMT workflow described herein is equally applicable to tracking and analyzing fluorescent target compounds. For example, but not by way of limitation, the compounds described herein may themselves be fluorescent or may be modified to facilitate fluorescent detection. Furthermore, changes in the motion of fluorescent compounds can be used to determine the SMT profile of the compound itself. Thus, all analytical strategies described herein for tracking a target fluorescent protein are also applicable to results obtained by tracking the compound itself.

[0128] 2.2.2. OLS Single Molecule Tracking Sample Preparation Referring to FIG. 5, embodiments of the present subject matter can be implemented using an htSMT workflow whereby cells (2-018) are seeded into a sample plate (2-021), e.g., a tissue culture-treated 384-well glass-bottom plate, although other plate types, including but not limited to single chamber, 9-well glass-bottom plates, 24-well glass-bottom plates, 96-well glass-bottom plates, 1536-well glass-bottom plates, and 3456-well glass-bottom plates, as well as plates made of alternative materials, e.g., plates made partially or entirely of plastic, can also be used in conjunction with the approach outlined herein. In certain embodiments, cells (2-018) are seeded at 1 to 20,000 cells per well (2-016), e.g., 50 to 10,000 cells, 100 to 9,000 cells, 250 to 8,500 cells, 500 to 7,500 cells, 750 to 7,000 cells, 2,500 to 6,500 cells, or 6,000 cells. The seeded cells can then be incubated under conditions favorable for attachment, e.g., overnight at 37°C and 5% CO2. Cells can be incubated with a sufficient amount of label to allow fluorescence, e.g., in the case of HaloTag fusions, 0.1 to 100 pM of JF 549 The desired results can be achieved by incubating -HTL (catalog number GA1110, Promega) with 50 nM Hoechst33342 (for labeling nuclei) in complete medium for 1 hour.

[0129] In certain embodiments of the htSMT strategy described herein, the cells are then washed, for example, three times in DPBS and twice in imaging medium. In certain embodiments, the imaging medium is prepared to facilitate fluorescence, for example, fluoroBright DMEM medium (catalog number A1896701, Thermo Fisher), which can be supplemented with GlutaMAX (catalog number 35050079, Thermo Fisher) and the same serum and antibiotics as the growth medium.

[0130] If appropriate, compounds can be added to samples to test their effect on specific fluorescent proteins via SMT. In certain embodiments, compounds can be serially diluted in an Echo-certified 384-well low-dead-volume source microplate (product number 0018544, Beckman Coulter) to generate source material for dose titration. Compounds can then be administered to cell culture media at a final dilution of, for example, 1:1000. In certain embodiments of the htSMT strategy described herein, each dose of compound has at least two replicates per plate and three plate replicates. Additionally, in certain embodiments of the htSMT strategy described herein, 20 DMSO control wells and two no-dye control wells can be randomized across each plate (2-012). In certain embodiments, compounds can be incubated for 0-48 hours, for example, 1 hour at 37°C, before acquiring images.

[0131] 3. htSMT software 3.1. htSMT Software Overview FIG. 6 illustrates an exemplary system 600 for a high-throughput single-molecule imaging platform for measuring molecular movement in living cells. An experiment 602 can be performed to collect large amounts of data from multiple living cells (e.g., using an imaging system 624 to identify compounds 626 and / or targets 622). The experiment 602 can include applying various identifiers, such as labels that can subsequently fluoresce or otherwise be detected, to molecules of interest (e.g., using a laser or other light source). Biological samples forming part of such an experiment 602 can be organized in a plate 604 having multiple wells 606. Each well 606 can have one or more associated fields of view (FOVs) 610. An FOV 610 can be a position within or corresponding to a single well 606. Image sequences can be generated for the FOV 610 to generate one or more movies 612, which can include SMT movies and non-SMT movies. SMT movies can be used to track the path of individual labeled molecules, such as proteins, generating multiple trajectories. Each trajectory may consist of multiple spots 614 containing spatiotemporal coordinates of labeled molecules at a particular time (as described in more detail in FIG. 7). Separately from, and in some examples in parallel with, tracking, the video 612 can be used to identify molecules by using machine learning and / or computer vision-based image segmentation to generate masks 618. Masks 618 are spatial regions within the FOV 610 generated by segmentation. Each mask 618 can belong to a mask category, which is described in more detail in FIG. 8.

[0132] Data associated with the two channels (e.g., the tracking channel and the segmentation / masking channel) can be combined to generate multiple metrics 620 associated with various aspects of the sample. In other words, the trajectories 616 (e.g., trajectory data) can be combined with the machine learning processed image segmentation data and further analyzed using statistical / machine learning techniques. Processing of the combined data can be used to generate metrics 620, such as hit scores, associated with compounds and / or targets within the biological sample, which may be stored in a database structure, as further described in FIG. 9.

[0133] FIG. 7 illustrates data flow through an exemplary system 700 for a high-throughput single-molecule imaging platform for measuring protein movement in live cells. An experiment specification 704 defining an experiment 602 can be provided as data input via one or more clients 702. For example, each experiment 602 can be collected along with an accompanying stain (e.g., Hoechst or Promac Red) used for downstream analysis, including segmentation 618. The experiment specification 704 can define various parameters for the experiment 602, such as stains, dyes, compounds, and treatments. As previously described in FIG. 6, the imaging system 706 (e.g., imaging system 624) can capture a sequence of images that generate one or more SMT movies 711 and / or non-SMT or segmented movies 708 (e.g., movie 612) that characterize the movement of molecules. The SMT movie 711 can characterize the movement of individual fluorophores and / or can include images of individual fluorophores. The segmented movie 708 can include a sequence of images that characterize the movement of a labeled molecule and / or its components. It will be understood that Hoechst staining is only one technique that may be used to label molecules, and / or multiple labeling techniques, such as Potomoc Red, may be utilized depending on the desired configuration. For example, MitoTracker Deep Red may be used to label mitochondria, Concanavalin A-dye conjugates may be used to label endoplasmic reticulum, SYTO14 may be used to label nucleoli, phalloidin may be used to label actin, etc.

[0134] The SMT movie 711 can be analyzed to perform operations related to molecule tracking 710, which can include detecting 712, sub-pixel localizing 713, and linking 714 to identify trajectories 715 of molecules across various images in the SMT movie 711. More specifically, during detecting 712, one or more spots can be detected or recovered in the SMT movie 711. Each spot can be provided with spatiotemporal coordinates. These spatiotemporal coordinates can be estimated by using sub-pixel localizing techniques 713. Linking 714 can be performed on the spots to ultimately identify trajectories 715.

[0135] As used herein, a link is a potential connection between two spots. Each link is directed, starting at one spot and ending at another. A "correct link" connects two spots generated by the same emitter in different frames; otherwise, the link is "incorrect." One goal of the linking algorithm is to estimate which link is correct. In this specification, links are referred to in the form a:i→j, which is interpreted to mean link a starting at spot i and ending at spot j. A link satisfies at least three of the following constraints: (a) the link moves forward in time; (b) the link cannot connect two spots that are more distant than a certain limit (hereinafter referred to as the "search radius"); and (c) the link cannot connect two spots that are more distant in time than a certain limit (hereinafter referred to as the "gap limit"). A spot-link graph is a graph of the spots and links of a single SMT movie 711. In this graph, spots are vertices and links are edges. Because links move forward in time, the spot-link graph is a directed acyclic graph. A matching is a subset of links in the spot-link graph, such that no two links in this subset start or end at the same spot. A trajectory 715 is used herein to refer to a sequence of consecutive (end-to-end) links in the same matching. A dynamic metric 730 can be determined using multiple trajectories. Such parameters can include spot attributes that characterize the spot's movement. Such parameters can include one or more of the velocity, diffusion coefficient, or anomaly parameter(s) of each spot. The dynamic parameter(s) of spot i are defined herein as θ i Herein, the set of dynamic parameters of all spots in the spot-link graph is called Θ.

[0136] Separately from, and in some variations in parallel with, the processing of the SMT video 711, the segmented video 708 can undergo segmentation to generate one or more masks 720. The masks can be classified into various categories, including, but not limited to, cell nuclei, cytoplasm, and / or extraneous masks, which are further described in FIG. 8 . An instance mask is an individually segmented object (e.g., one cell, one nucleus, one mitochondrion). The FOV 610 can include any number of instance masks for one mask category. A semantic mask is the union of all instance masks corresponding to one type of mask category for one FOV (e.g., all cells, all nuclei, or all mitochondria for one FOV). Extraneous masks can include portions of the non-SMT video 708 that are excluded from any downstream data analysis. For example, these extraneous masks can correspond to portions of the non-SMT video 708 that are out of focus or contain autofluorescent cellular debris that prevents accurate tracking. During segmentation, molecules in the segmented movie 708 can be assigned to one or more masks. Image metrics 740 can be assessed from the masked molecules, such as cell health, focus quality, etc.

[0137] Experiment information, such as dynamic metrics 730, image metrics 740, and any data from which any metrics are derived (e.g., segmentation information), may be provided to a data repository 770 for storage. Such a data repository 770 may store any results of the experiment 602, such as, for example, dynamic metrics 730, image metrics 740, and / or any data from which any metrics are derived. The data repository may include a local persistence server and / or a dedicated server accessed locally or via the cloud. The data repository 770 may also store metadata associated therewith and / or associated with the experiment specification 704. Experiment information (e.g., results and metadata from past experiments, etc.) may be provided to the data repository 770 via a repository application program interface (API) 750. The repository API 750 may also interface with a web-based graphical user interface front end 760 that provides such information for display on the client 702.

[0138] In some variations, the segmentation information can be used to identify subcellular compartments such as the nucleus, nucleolus, cytoplasm, etc. The segmentation information may also be used to distinguish one cell from another. The segmentation information may be stored in a particular format (e.g., a multi-image file format such as TIFF, etc.).

[0139] The exemplary dynamics metric 730 can also include a state array. The state array is a framework for learning interpretable dynamic models from SMT trajectories and can be used to gain additional insight into the movement of a target protein and where that movement occurs within the cell. In some variations, the state array can be generated / added using segmentation information. The state array output can be returned at the subcellular compartment level, allowing researchers to distinguish between dynamics in different subcellular compartments. Additionally, the state array can be calculated for each individual subcellular compartment (e.g., for each nucleus).

[0140] To facilitate data access by applications, including but not limited to state arrays, processed SMT data may be stored in a format that allows for (a) representation of processed trajectories and associated attributes, such as the SNR and spot shape characteristics of each SMT video; (b) representation of mask objects, including mask categories (e.g., the subcellular organelles associated with each mask object); (c) association of trajectories with mask objects (e.g., the cell nuclei in which each trajectory was observed); and (d) association of all SMT videos with metadata related to the original experiment, such as compound treatment, acquisition time, and imaging system name. Formats (a) and (c) may be protocol buffer schemas that define the storage format for trajectories and associated mask objects. Format (b) may be a specialized image file format containing the mask object to which each pixel in the FOV belongs. Format (d) may be a PostgreSQL database that records all captured experiments / videos. As a client of the processed SMT data, state arrays can reference these data schemas to report the dynamic properties of trajectories by mask category or by mask object.

[0141] FIG. 8 illustrates a plurality of images 800 illustrating the distinction between mask categories and instance or semantic masks. As previously described, a non-SMT video or segmented video can be assigned to multiple categories. Such categories may include cell nuclei (e.g., Category A), cytoplasm (e.g., Category B), and / or extraneous masks (e.g., Category C). Unique, individual masks can be applied to a biological sample. For example, image 810 illustrates a unique, individual instance mask applied to a cell nucleus (e.g., Category A). Image 812 illustrates a unique, individual instance mask applied to a cell cytoplasm (e.g., Category B). Image 820 illustrates multiple instance masks applied to one or more nuclei, each with a different, unique, individual instance mask. Image 822 illustrates multiple masks applied to one or more cytoplasms, each with a different, unique, individual instance mask. Image 830 illustrates a semantic mask that is the union of all instance masks applied to one or more nuclei. Image 832 illustrates a semantic mask applied to one or more cytoplasms.

[0142] FIG. 9 illustrates an exemplary computer-implemented environment 900 in which an imaging system 910 can interact with a computing architecture to execute various algorithms described herein. As shown in FIG. 9 , the imaging system 910 can interface with one or more clients 950 (e.g., client 702 via a web application having a graphical user interface). The one or more clients 950 can interface with one or more servers 920 accessible via network(s) 930. The one or more clients 950 can host a frame grabber that captures images (e.g., video 612) from a camera. These images can be temporarily stored on the one or more clients 950 and periodically transferred via the network 930 to the one or more servers 920 for remote storage. The one or more servers 920 also include or have access to one or more data stores 940 for storing data collected and / or extracted from the sample by the imaging system 910. In some variations, the network 930 may include or interface with one or more network storage arrays 960 for storing data such as captured images (e.g., video 612).

[0143] FIG. 10 is a diagram 1000 illustrating a sample computing device architecture for implementing various aspects described herein. In some variations, the sample computing device architecture may be that of a client(s) 950 and / or a server(s) 920, and some components described in connection with diagram 1000 may be optional for the client(s) 950 and / or the server(s) 920. A bus 1004 may function as an information highway interconnecting the other illustrated components of hardware. A processing system 1008 (e.g., one or more computer processors / data processors in a given computer or in multiple computers), labeled CPU (Central Processing Unit), may perform the computational and logical operations required to execute a program. Optionally, or additionally, a processing system 1012 (e.g., one or more computer processors / data processors in a given computer or in multiple computers), labeled GPU (Graphics Processing Unit), may perform the computational and logical operations required to execute a program. Non-transitory processor-readable storage media, such as read-only memory (ROM) 1016 and random access memory (RAM) 1020, may be in communication with processing system 1008 and / or processing system 1012 and may contain one or more programming instructions for the operations specified herein. Optionally, the program instructions may be stored on a non-transitory computer-readable storage medium, such as a magnetic disk, optical disk, recordable memory device, flash memory, solid-state drive, or other physical storage medium.

[0144] In one example, the disk controller 1048 can interface with one or more optional removable storage 1056 or local storage 1052 via the system bus 1004. The removable storage 1056 can be an external or internal disk drive, a solid-state drive, or an external hard drive. The local storage 1052 can be an internal hard drive and / or memory. As mentioned above, these various examples of the removable storage 1056, the local storage 1052, and the disk controller 1048 are optional devices. The system bus 1004 may also include at least one communication interface 1024 to enable communication with external devices either physically connected to the computing system or externally available via a wired or wireless network, such as cloud storage or a remote service. In some cases, the at least one communication interface 1024 includes or otherwise comprises a network interface.

[0145] In some variations, e.g., client(s) 950, to provide for interaction with a user, the subject matter described herein can be implemented on a computing device having a display device 1044 (e.g., an LCD (liquid crystal display) or LED (light emitting diode) monitor) for displaying information retrieved from bus 1004 to a user via display interface 1040, and input devices 1032, such as a keyboard and / or pointing device (e.g., a mouse or trackball) and / or a touch screen, for a user to provide input to the computer. Other types of input devices 1032 can also be used to provide interaction with a user. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback via microphone 1036, or tactile feedback), and input from the user can be received in any form, including acoustic, voice, or tactile input. The input devices 1032 and microphone 1036 can be connected to bus 1004 via input device interface 1028 to communicate information. As an example, the input device 1032 may be the imaging system 910 configured with the capability to capture a sequence of images, as described herein. The frame grabber 1058 may capture or grab individual frames from analog or digital data encapsulating the sequence of images acquired from the bus 1004. The frame grabber 1058 may include memory capable of storing single or multiple frames. The frame grabber 1058 may also provide individual frames or multiple frames to the bus 1004 for further storage, for example, in the local storage 1052 and / or the removable storage 1056. Other computing devices, such as dedicated servers, may omit one or more of the components described in connection with FIG. 10 .

[0146] One or more aspects or features of the subject matter described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof. These various aspects or features may include implementation in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be special-purpose or general-purpose, coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and by virtue of the client-server relationship they have to each other.

[0147] These computer programs, which may also be referred to as programs, software, software applications, applications, components, or code, contain machine instructions for a programmable processor and may be implemented in high-level procedural languages, object-oriented programming languages, functional programming languages, logic programming languages, and / or assembly / machine languages. As used herein, a "machine-readable medium" refers to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD), etc.) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor. A machine-readable medium may non-transitory store such machine instructions, such as, for example, a non-transitory solid-state memory, a magnetic hard drive, or any equivalent storage medium. Alternatively or additionally, a machine-readable medium may temporarily store such machine instructions, such as, for example, a processor cache or other random access memory associated with one or more physical processor cores.

[0148] 3.2. Probabilistic methods for high-density molecular tracking in living cells One aspect of htSMT workflows 1-100 involves the recovery of trajectories 715, or paths of identifiers (e.g., individual fluorescent emitters, etc.), from a recorded image sequence (e.g., SMT movie 711). This recovery, as described herein, is "tracking."

[0149] There are several challenges associated with htSMT tracking 710. First, each emitter is dim, contributing only 100 photons per frame, which may require highly sensitive detection methods. Second, the absolute intensity and noise characteristics of each video may depend on the original imaging system. These differences may arise from differences in laser power or camera gain and offset. As a result, htSMT tracking methods are desirable to be invariant to changes in the absolute intensity of the video. Third, protein movement within cells is fast, and molecules can move quickly in and out of focus. As a result, the average trajectory length is short, only 3–4 frames, which can severely limit the information available for predicting the molecule's future movement. Fourth, tracking becomes difficult at high label densities due to ambiguity in relating detections to trajectories. For example, tracking methods can modulate their parameters in a density-dependent manner to achieve accurate tracking at various densities.

[0150] As previously described, htSMT tracking 710 can include detection 712, sub-pixel localization 713, and linking 714. First, during detection 712, portions of each video frame of the SMT video 711 containing emitters are identified. Second, sub-pixel localization 713 infers emitter locations with sub-pixel resolution, resulting in spatiotemporal coordinates for each detected emitter. For example, sub-pixel localization 713 may fit the observed light distribution around the emitter to an approximation of the point spread function (PSF) of the imaging system. Third, a linking algorithm associates detected emitters with trajectories. While all steps must be performed to meet the needs of htSMT data processing, linking 714 in particular can become prohibitively expensive as label density and / or the number of detections per frame increase. This can be due to the combinatorial explosion of possible trajectories that can be constructed from a given set of detections.

[0151] Figure 11 shows a diagram 1100 illustrating a pipeline 1110 for scalable tracking in htSMT. Diagram 1100 is a modular structure that allows custom combinations of the tracking 710 (e.g., detection 712, sub-pixel localization 713, and linking 714) described in Figure 7 at runtime as specified in an experiment-specific configuration file 1122. Each tracking pipeline 1110 can have an associated runtime configuration 1120 specified by the experiment-specific configuration file 1122, which can define particular types of detectors 1112, sub-pixel localizers 1113, and linkers 1114.

[0152] The tracking pipeline 1110 may receive as input an image sequence (e.g., an SMT video) 1111. To detect or retrieve one or more spots in the image sequence 1111, a detector 1112 may be applied to the image sequence 1111 using any of the following detector types: a generalized log-likelihood ratio spot detector, a difference of Gaussians (DoG) detector, a Laplacian of Gaussians (LoG) detector, a Hessian determinant (DoH) blob detector, or any combination thereof. Depending on the implementation, other types of detectors may be used.

[0153] The detected spots can be associated with spatiotemporal coordinates using sub-pixel localization 1113. Such sub-pixel localization 1113 can include any of the following localizer types: radially symmetric localizer, maximum likelihood fitting to a candidate spot model using the Levenberg-Marquardt method, etc. Depending on the implementation, other types of localization techniques can also be used.

[0154] Linking of two spots can be performed using a linker 1114. In some variations, the linker 1114 can rely on either heuristics (e.g., nearest neighbor methods) or exact solutions to the assignment problem (e.g., the Hungarian algorithm). Such linker types can utilize separate steps for inferring trajectories and inferring dynamic parameters from the trajectories. In other variations, the linker 1114 can infer a joint probability distribution over possible trajectories and dynamic parameters in a scalable manner. This distribution can be used to make a more informed point estimate of the "correct" trajectory, to estimate the confidence of a particular set of trajectories, or to derive dynamic results completely independent of the trajectories. This approach, referred to herein as "probabilistic linking," can be used to estimate distributions over trajectories in videos containing thousands to tens of thousands of nearby fast-moving targets.

[0155] The tracking pipeline 1110 can output object tracks 1115 in a graphical format that represent possible trajectories. This output can depend on the type of linker utilized by the linker 1114.

[0156] 3.2.1. Probabilistic methods for high-density molecular tracking in living cells Two exemplary probabilistic link types of linker 1114 may utilize different methods, including variational Bayesian inference (referred to herein as "vtrack") or Gibbs sampling (referred to herein as "gibbstrack").

[0157] The dynamic parameters of each spot can be viewed as arguments to a motion model that defines a probability distribution over its future motion. Here, any vector displacement r between spots i and j i→j We can assume that the probability of depends only on the dynamic parameters of spots i and j, and not on the rest of the spot-link graph. Equation 1 expresses this probability and defines the "motion model". f r|θ(r i→j │θ i ,θ j ) (1)

[0158] One alternative to Equation 1 is the likelihood function for scaled Brownian motion, which can be characterized by a single kinetic parameter (the diffusion coefficient) per spot. A simplified Bayesian model for scaled Brownian motion is detailed in Section 3.2.6 below.

[0159] The objective function of the link algorithm may be defined as follows: Let M be the number of links in the spot link graph. E∈{0,1} M If link a participates in the matching, then E a = 1, otherwise E a Let w∈R be a vector of 1s and 0s that represent matchings such that =0. M Let be a real-valued weight vector. Let the likelihood of starting or ending a trajectory be w0. Then, the role of the link algorithm is to find the best matching

number

number

[0160] Equation 2 (eg, the solution criterion for the link problem) is an example of an imbalanced assignment problem because each matching cannot contain two links that start or end at the same spot.

[0161] Since each element of the matching vector E corresponds to a link a:i→j, in this specification, each element is represented by its link index a (i.e., E a ) or its spot index (i.e., E i→j ) for convenience. Similarly, vector displacements can be indexed by r a or r i→jcan be expressed as either:

[0162] If the weight vector is constant, Equation 2 can be solved by classical methods for solving assignment problems. These include exact methods such as the Hungarian algorithm and heuristics such as nearest neighbor methods.

[0163] In general, however, the weight vector can be a function of a dynamic parameter Θ, which can be estimated from the trajectory defined by the matching vector E. Since both E and Θ are unknown a priori, they can be jointly estimated.

[0164] The goal of the probabilistic link algorithm is to estimate the conditional distribution p(E,Θ│R), where R = (r1,…,r M ) represents the vector displacements corresponding to each of the M links in the spot-link graph. Optionally, these terms may be generalized to include any additional information relevant to the link problem (spatial location, spot shape characteristics, etc.). Once the distribution p(E,Θ|R) is obtained (e.g., via vtrack or gibbstrack as discussed herein), this distribution can be used to estimate the maximum a posteriori trajectory, or the dynamic parameters, by solving Equation 2, with the marginal link probabilities w=logp(E|R).

number

[0165] Bayes' theorem allows us to write the conditional distribution p(E,Θ|R) as Equation 3 (e.g., Bayes' theorem for the linking problem).

number

[0166] In Equation 3, the term p(E,Θ|R) is the likelihood of the observed displacement given the dynamic model Θ and the matching vector E. r|θ (r i→j │θ i ,θ j ) is the dynamic parameter θ i and θ j A single displacement r given i→j Since the likelihood function of is (Equation 1), the total likelihood function p(R|E,Θ) is given by the product of the likelihood functions of each link (Equation 4). In Equation 4 (e.g., the likelihood function for the link problem), Pa(i) is the set of parent spots of spot i, which includes the set of all spots j such that j → i is a allowed link.

number

[0167] In Equation 3, the term p(E, Θ) is a prior distribution on E and Θ. Herein, we can assume that p(E, Θ) = p(E)p(Θ), where p(E) = constant for all allowed E,

number

[0168] In Equation 3, the term p(R) is the so-called "evidence" and can be analytically intractable. As a result, the left-hand side of Equation 3 cannot be evaluated in closed form. The methods described herein, based on Gibbs sampling (gibbstrack) and variational track (vtrack), circumvent this problem. Gibbstrack approximates the left-hand side of Equation 3 by drawing a fixed number of random samples, while vtrack approximates the left-hand side of Equation 3 using a mean-field approximation. Gibbstrack and vtrack are each described in detail below.

[0169] 3.2.2. Probabilistic Linking via Variational Bayesian Optimization (vtrack) Using vtrack, the posterior distribution p(E,Θ|R) can be approximated using variational Bayesian optimization. In this method, by assuming that the posterior distribution is factored in E and Θ, we approximate:

number

[0170] By working with the model introduced in Equation 4, a simplified version of vtrack can be derived. This derivation allows for greater freedom in the choice of motion model, as vtrack can accommodate a variety of motion models. As shown below, vtrack can be derived for the Brownian motion model or for more general models.

[0171] We consider the joint probability function over all variables in the model, represented by Equation 4. With this choice of prior distribution, as explained in Section 3.2.1, the joint probability distribution over all parameters can be factored in the form presented by Equation 5 (e.g., the joint probability density for the linking problem). p(R,E,Θ)=p(R│E,Θ)p(E)p(Θ) (5)

[0172] Substituting Equation 4 and the prior distribution p(R, Θ) into Equation 5 and taking the logarithm, we obtain Equation 6 (e.g., the logarithmic joint probability density for the link problem).

number

[0173] The variational pursuit approach finds an analytical approximation to the true posterior distribution that satisfies two criteria:

number

[0174] Then, q maximizes the lower bound on the evidence (Equations 8 and 9).

number

number

[0175] Any distribution q(E,Θ) that satisfies Equations 7 and 9 must be Equations 10 and 11. In Equation 10 (e.g., the recurrence equation for the factors of q(E)) and Equation 11 (e.g., the recurrence equation for the factor q(Θ)), logp(R,E,Θ) is given by Equation 6, and the expected value

number

number

number

number

[0176] Equations 10 and 11 can be solved sequentially to produce a progressively better approximation q(E,Θ), a scheme known as expectation-maximization. This algorithm converges because the lower bound on the evidence (Equation 8) is convex with respect to each factor in q. This method is referred to herein as vtrack, in reference to the tracking model defined by Equation 6.

[0177] Once q(E,Θ) is obtained, we convert the weight vector into the marginal log-probability of each link.

number

number

[0178] Equations 10 and 11 may be solved for any motion model for which there exists a conjugate prior (i.e., any choice of Equation 1), in combination with the log probability density represented by Equation 6. This includes any motion model with a probability density in the exponential family of distributions.

[0179] 3.2.3. Specific Form of vtrack of Brownian Motion As a limited demonstration of vtrack, we can now solve Equations 10 and 11 for the Brownian motion model expressed in Equation 19 using the prior expressed in Equation 20. Under these conditions, the log joint probability (Equation 6) becomes Equation 12 (e.g., the joint probability density of Brownian motion). In Equation 12, m is the spatial dimension and θ i is the diffusion coefficient of spot i, and r i→j is the spatial displacement of link i→j, Δt is the frame interval, and α0 and β0 are a priori parameters.

number

[0180] Substituting Equation 12 into Equation 10 and Equation 11 and solving for the respective factors q(E) and q(Θ), we obtain Equation 13 and Equation 14. In these equations, GraphSoftmax is the graphical softmax operator described below, T is temperature, w0 is the trajectory initiation likelihood, and

number

number

number

number

number

number

number

number

number

number

[0181] 3.2.4. Extended Variational Pursuit Algorithm The vtrack algorithm above can be generalized to exploit the additional information implicit in the spot-link graph as follows: Given a spot-link graph containing N spots and M links,

number

number

[0182] As in the case of the simple vtrack algorithm, an approximate posterior distribution

number

[0183] As an example of a solution, the results can be demonstrated using Brownian motion. To do this, f r|θ (r i→j │θ i ) is a gamma distribution of the form specified in Equation 19, and p(θ i ) can be assumed to be an inverse gamma distribution of the form specified in Equation 19. The posterior distribution specified in Equation 16 and Equation 17 can be obtained. In Equation 16, the term

number

number

number

number

number

number

[0184] As in the case of the simple vtrack algorithm, once an approximate posterior distribution q(E,Θ) is obtained, the log-marginal link probabilities (

number

[0185] 3.2.5. Gibbs Sampling for Probabilistic Links (Gibbstrack) Another approach to estimating p(E,Θ|R) is to draw random samples from this distribution and then take the average of these samples to approximate the mean of the posterior distribution. A simple and flexible way to achieve this sampling scheme is to alternately sample from the conditional distributions of E and Θ (Equation 18). Equation 18 defines Gibbs tracking and can be expressed as follows: E~p(E│R,Θ)(18) Θ~p(Θ│R,E)

[0186] The sampling scheme expressed in Equation 18 can be achieved as follows: Start with an estimate of the dynamic parameters Θ and set E to all zeros (this is always a valid matching vector for any spot-link graph). At each iteration, evaluate the log-likelihood of each link according to the current dynamic model (via Equation 1 for selecting the motion model).

number

number

[0187] E1,E2,…,E n and Θ1, Θ2,…, Θ n If is a sample generated in this way from Equation 18, then the peripheral link probability

number

number

number

[0188] 3.2.6. Bayesian Model of Brownian Motion One type of motion model for Vtrack and Gibbstrack is scaled Brownian motion, which characterizes the motion of each spot with a single dynamic parameter (the diffusion coefficient). Under scaled Brownian motion, the likelihood function (Equation 1) becomes a gamma distribution, expressed as Equation 19, where m represents the dimension of the space in which the motion is observed, and θ i is the diffusion coefficient of spot i, and r i→jis the vector displacement corresponding to link i → j. Equation 19 is the likelihood function of Brownian motion in m dimensions, which can be expressed as:

number

[0189] A useful prior distribution conjugate to the Brownian likelihood expressed in Equation 19 is the inverse gamma prior, expressed in Equation 20, where α0 and β0 are prior hyperparameters, Δt is the frame interval, and θ is the diffusion coefficient of a single spot. Equation 20 is the prior distribution for Brownian motion and can be expressed as follows:

number

[0190] A series of observed displacements r1, r2, ..., r n Given f, the posterior distribution of the diffusion coefficient is given by Equation 21, which forms the basis for Bayesian inference of the diffusion coefficient. θ refers to the inverse gamma distribution of the form given by Equation 20. Equation 21 is the posterior distribution of Brownian motion, which can be expressed as:

number

[0191] 3.2.7. Sparse Hill Climbing Algorithm Equation 3 is a weight vector, under the condition that the weights are constant.

number

[0192] In the subsequent algorithm, the "pivot" is a matching vector E∈{0,1} that modifies at most four elements, as follows: M Each pivot is a weight change Δw=W that determines whether the pivot is accepted. a -W b -W c +W d Each pivot selects a link a:i → j and a =w a It is started by setting E b If there exists a link b:i→k such that =1, then W b =w b otherwise, W b = w0. E c If there exists a link c:l→j such that =1, then W c =w c otherwise, W c If both of the above conditions are true and d:i→k is a link, then W d =w d otherwise, W d = w0. The pivot is Δw = W a -W b -W c +W d > 0 is accepted. If the pivot is accepted, E a = 1, if b is a link, E b =0, if c is a link, E c = 0, if d is a link, E d =1 to perform a pivot.

[0193] The lookup required at each pivot (e.g., determining whether links b, c, and d exist) can be performed quickly by keeping in memory a record of each spot's currently assigned forward and reverse links.

[0194] The sparse hill climbing algorithm may be described as follows for a spot-link graph with N spots and M links: M and the known weight vector

number

[0195] One way to determine convergence is to check that there is no change in the matching vector E after iterating through all M links in random order.

[0196] The sparse hill-climbing algorithm provides a nearest-neighbor solution to the tracking problem (for each link a:i → j, a =w0-r i→j (r i→j It can be used for a variety of purposes, including estimating σ(σ), ...

[0197] Graphical Softmax One aspect of the vtrack algorithm is to normalize some link log-likelihoods for the incoming and outgoing links in the spot-link graph, taking into account dependencies between links induced by topological constraints on matching (e.g., two links in the same matching cannot start or end at the same spot).

[0198] One way to deal with regularization is to generalize the softmax operator (e.g., the Boltzmann distribution) to doubly stochastic matrices.

[0199] Another way to deal with normalization is to use the "graphical softmax" operator. The input to the graphical softmax operator is a spot-link graph with N spots and M links, where each link

number

number

number

[0200] Five vectors,

number

number

number

number

number

number

number

[0201] The start probability v and end probability w are the probabilities of all links entering and leaving each spot.

number

[0202] The convergence rate of this algorithm depends more on the sparsity of the problem than on its size: for htSMT, convergence can occur in about 20 iterations.

[0203] 3.2.9. Measures of confidence in tracking solutions Ensuring high-quality data can increase confidence in the results produced by an htSMT system. However, because htSMTs can be generated rapidly and continuously across multiple imaging systems, the ability of human supervision to uncover problems in the data can be limited. Therefore, a feature of the tracking pipeline described herein is that it provides a built-in measure of confidence in the tracking solution that can be used as a diagnostic for imaging quality in lieu of direct human supervision.

[0204] Equation 22 is the normalized entropy of the posterior distribution over the trajectories, which can be defined as:

number

number

[0205] Equation 22 defines the confidence in the solution of the tracking algorithm, with values ​​closer to 0 indicating higher confidence. Values ​​below 0.4 may reflect a high degree of confidence in the tracking solution. However, tracking particles at very high speeds (especially in the case of Brownian motion) can be prone to errors, and a higher threshold may be acceptable for such use cases.

[0206] The tracking error rate lower bound (ERLB) is a second diagnostic useful for assessing the quality of hSMT results. This metric is designed as a lower bound on the fraction of incorrect links created by the tracking algorithm. For example, a value of 0.1 indicates that at least 10% of the links created by the tracking algorithm are likely to be incorrect. Because it may not be known in advance which links are correct or incorrect, ERLB creates a situation where a subset of links is known to be incorrect.

[0207] The ERLB is computed once per SMT video: the set of detections from the second half of the video is superimposed on the set of detections from the first half of the video. The tracking algorithm is then re-run on this superimposed set of detections, without knowledge of which detections come from which half of the video. Under these conditions, any link created by the tracking algorithm between two detections derived from different halves of an SMT video will be incorrect. Since it may be unclear whether a link between two detections derived from the same half of the video is incorrect, the proportion of such links is a lower bound on the error rate.

[0208] The process of overlapping the two halves of the video is achieved in the following way: Each spot is always associated with a frame index, or the index of the image in the original sequence of images from which the spot was obtained. Let S1 be the set of spots from the first half of the SMT video, and S2 be the set of spots from the second half of the video. Let T be the total number of frames in the video. For each spot in S2, subtract floor(T / 2) from its original frame index. Then, combine S1 and S2 into a new set of detections.

number

[0209] 3.2.10. Trajectory-Independent Dynamic Estimation Using Probabilistic Tracking Algorithms The goal of htSMT is to infer the dynamic parameters of a target protein and identify experimental conditions that alter its dynamics. For example, one may estimate the diffusion coefficient of a target protein for each of several compound treatments to identify compounds that perturb the protein's diffusion state (e.g., by creating or disrupting protein-protein interactions).

[0210] These dynamic parameters are typically estimated from the trajectories. However, stochastic tracking provides a posterior distribution over both the dynamic parameters and the trajectories p(E,Θ|R), so that the estimation of the dynamic parameters can be done by marginalizing over the trajectories.

number

number

[0211] As an example, we describe a procedure for estimating the marginal posterior diffusion coefficients over all possible past and all possible future times for each spot.

number

[0212] n SMT videos with M possible links and N spots.

number

number

number

number

number

number

number

[0213] 3.2.11. Tracking Algorithm Benchmark Figures 12A-12C show benchmarks of various tracking algorithms. Regarding Figure 12A, optical dynamic simulations were used to test the accuracy of several linking algorithms, including vTrack, GibbsTrack, and adaptive hill climbing, as well as other linking algorithms: random, conservative, and nearest neighbor. In the random linking algorithm, which served as a control, each detection was randomly linked to another detection within its distance gate (i.e., the set of detections within its search range and gap limit). The conservative linking algorithm was configured so that each detection was linked to another detection only if there were no other possible detections within the applicable distance gate. The nearest neighbor linking algorithm stipulated that each detection was linked to its nearest neighbor within the applicable distance gate. Three classes of experiments (Benchmark 1, Benchmark 2, and Benchmark 3) were conducted with increasing difficulty. The outputs of these experiments were link recall, link precision, and F1 score (i.e., the harmonic mean of recall and precision). The metrics in this context are shown in Figure 12B. To ensure a fair comparison, the search range (i.e., the maximum distance over which links are considered) and gap limit (i.e., the maximum number of gap frames over which links are considered) were kept constant at 1.25 μm and 2 gaps for all algorithms. The exception is the conservative linking algorithm, which essentially requires 0 gaps (so all links are between consecutive frames). The benchmark results are shown in Figure 12C.

[0214] 4. Specific htSMT Applications Many, perhaps most, pathways that regulate fundamental cellular biochemistry rely on the interaction of protein sensors and protein effectors that transiently engage and trigger changes in cellular physiology. While the fundamentals of this process have long been recognized, biochemical investigation of these protein interactions has typically required in vitro reconstitution or been investigated through pull-down assays after cell permeabilization. The htSMT workflow described herein provides a means to visualize protein movement in large numbers of live cells, and also in contexts where the effects of additive compounds, such as small molecule inhibitors, can be quantitatively assessed.

[0215] 1, aspects of the htSMT workflow of the present disclosure include, but are not limited to, (i) sample preparation, including reagent handling, (ii) image acquisition, imaging the sample to generate a series of images and / or video, (iii) image analysis, processing these images and videos, (iv) information storage, and (v) providing insights using the stored information, including biological interpretation. With respect to biological interpretation, the htSMT workflows described herein provide the ability to provide specific insights, as outlined below, depending on the particular workflow employed, e.g., (i) htSMT screening, (ii) htSMT binding, and / or (iii) kinetic SMT.

[0216] In certain embodiments, the disclosed workflow may include illuminating a detection field of view at a sample plane disposed within the sample with a light beam to induce fluorescence by a subset of fluorescent target proteins in live cells, thereby imaging a plurality of molecular trajectories. Here, the detection field of view has a size of a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm). For example, but not by way of limitation, the detection FOV may have a size of a first dimension (about 94 μm) by a second dimension (about 94 μm).

[0217] In certain embodiments, the disclosed workflow may include illuminating a detection field of view at a sample surface disposed within the sample with a light beam to induce fluorescence by a subset of fluorescent target proteins in live cells, thereby imaging a plurality of trajectories. In certain embodiments, the number of trajectories imaged within the detection field of view may be about 10,000 to about 100,000, e.g., about 20,000 to about 40,000. For example, and without limitation, the number of trajectories imaged within the detection field of view may be about 10,000 to about 50,000, about 10,000 to about 40,000, about 20,000 to about 50,000, or about 20,000 to about 40,000. In certain embodiments, the number of trajectories imaged within the detection field of view can be up to about 100,000, e.g., up to about 95,000, up to about 90,000, up to about 85,000, up to about 80,000, up to about 75,000, up to about 70,000, up to about 65,000, up to about 60,000, up to about 55,000, up to about 50,000, up to about 45,000, up to about 40,000, up to about 35,000, or up to about 30,000.

[0218] In certain embodiments, the detection field can include multiple cells. In certain embodiments, the number of cells imaged within the detection field is related to the size of the cells being imaged. For example, but not by way of limitation, the smaller the cell size, the greater the number of cells that can be imaged within the detection field. In certain embodiments, depending on the size of the cells being imaged, the detection field can include about 1 to about 50 live cells, e.g., about 1 to about 45 cells, about 1 to about 40 cells, about 1 to about 35 cells, about 1 to about 30 cells, about 1 to about 25 cells, about 1 to about 20 cells, about 1 to about 15 cells, about 1 to about 10 cells, about 1 to about 5 cells, about 5 to about 40 cells, about 10 to about 40 cells, about 15 to about 40 cells, about 20 to about 40 cells, about 10 to about 35 cells, about 15 to about 35 cells, or about 20 to about 30 cells. In certain embodiments, depending on the size of the cells being imaged, the detection field of view can include about 1 to about 40 live cells, e.g., mammalian cells. In certain embodiments, depending on the size of the cells being imaged, the detection field of view can include about 1 to about 30 live cells, e.g., mammalian cells. In certain embodiments, depending on the size of the cells being imaged, the detection field of view can include about 1 to about 20 live cells, e.g., mammalian cells. In certain embodiments, the detection field of view can include up to about 50 live cells, e.g., up to about 45 live cells, up to about 40 live cells, up to about 35 live cells, up to about 30 live cells, up to about 25 live cells, or up to about 20 live cells. In certain embodiments, the detection field of view can include up to about 40 live cells. In certain embodiments, the detection field of view can include up to about 30 live cells.

[0219] In certain embodiments, a workflow of the present disclosure can include detecting fluorescence from individual fluorescent target proteins in a plurality of fluorescent target proteins within a detection field of view at a sample plane at a rate of more than 100 detection FOVs per day, more than 10,000 detection FOVs per day, or more than 100,000 detection FOVs per day, where the detection field of view is a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm). In certain embodiments, a workflow of the present disclosure can include detecting fluorescence from individual fluorescent target proteins in a plurality of fluorescent target proteins within a detection field of view at a sample plane at a rate of more than 100 detection FOVs per day, where the detection field of view is a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm). In certain embodiments, a workflow of the present disclosure can include detecting fluorescence from individual fluorescent target proteins in a plurality of fluorescent target proteins within a detection field of view at a sample plane at a rate of more than 10,000 detection FOVs per day, where the detection field of view is a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm). In certain embodiments, a workflow of the present disclosure can include detecting fluorescence from individual fluorescent target proteins in a plurality of fluorescent target proteins within a detection field of view at a sample plane at a rate of more than 100,000 detection FOVs per day, where the detection field of view is a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm).

[0220] In certain embodiments, the disclosed workflow may include illuminating a detection field of a sample plane disposed within the sample with a light beam to induce fluorescence by a subset of fluorescent target proteins in live cells, thereby imaging a plurality of molecular trajectories. The detection field has a size of a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm), and sufficient laser illumination to track protein motion is achieved over up to 70% of the detection field. In certain embodiments, the detection field has a size of a first dimension (about 50 μm to less than about 100 μm) by a second dimension (about 50 μm to less than about 100 μm), and sufficient laser illumination to track protein motion is achieved over up to 60% of the detection field. In certain embodiments, the detection field has a size of about 50 μm to less than about 100 μm in a first dimension by about 50 μm to less than about 100 μm in a second dimension, and sufficient laser illumination to track protein motion is achieved over up to 50% of the detection field. In certain embodiments, the detection field has a size of about 50 μm to less than about 100 μm in a first dimension by about 50 μm to less than about 100 μm in a second dimension, and sufficient laser illumination to track protein motion is achieved over up to 40% of the detection field. In certain embodiments, the detection field has a size of about 50 μm to less than about 100 μm in a first dimension by about 50 μm to less than about 100 μm in a second dimension, and sufficient laser illumination to track protein motion is achieved over up to 30% of the detection field. In certain embodiments, the detection field has a size of about 50 μm to less than about 100 μm in a first dimension by about 50 μm to less than about 100 μm in a second dimension, and sufficient laser illumination to track protein motion is achieved over up to 20% of the detection field. In certain embodiments, the detection field has a size of about 50 μm to less than about 100 μm in a first dimension by about 50 μm to less than about 100 μm in a second dimension, and sufficient laser illumination to track protein motion is achieved over up to 10% of the detection field.

[0221] In certain embodiments, the disclosed workflow can include determining a change in the movement of a fluorescently labeled target protein in the presence of a compound. For example, but not limited to, the average change in the movement of the fluorescent target protein in the presence of the compound is at least 1%, at least 5%, or at least 10% compared to the change observed in the absence of the compound. In certain embodiments, the average change in the movement of the fluorescent target protein in the presence of the compound is about 1% to about 5%. In certain embodiments, the average change in the movement of the fluorescent target protein in the presence of the compound is about 1% to about 10%.

[0222] In certain embodiments, an exemplary htSMT workflow includes the individual strategies described above, as well as combinations of these strategies where two or more of the strategic requirements are combined.

[0223] 4.1.htSMT Screening In certain embodiments of the htSMT workflows described herein, the systems and methods are adapted to examine the ability of one or more compositions (e.g., "test" compounds) to affect the SMT profile associated with a fluorescent protein. For example, such htSMT workflows screen for changes in the SMT profile, e.g., either an increase or decrease in the movement of the protein of interest, in the presence of a composition compared to the SMT profile in the absence of the composition. It will also be appreciated that higher-order comparisons can be performed when compounds are multiplexed, including when multiple proteins are fluorescent. Furthermore, as outlined above, the htSMT screening strategies described herein are equally applicable to screening SMT profiles associated with fluorescent compounds, e.g., compounds that are naturally fluorescent or compounds that have been modified to fluoresce or linked to a fluorophore.

[0224] Fundamental to such an htSMT screening strategy is the ability of the htSMT workflow described herein to extract accurate kinetic data at scale. This capability is demonstrated by the exemplary results presented in Figures 13A-13E. In the experiments reported in Figures 13A-13E, a 384-well plate was used, with equal volumes of free Halo, Halo-CaaX, and H2B-Halo cell lines mixed in each well. Imaging using a 94 μm × 94 μm field of view (FOV) achieved an average of 10 nuclei simultaneously (Figures 13B, 19B), sufficient for most FOVs to contain cells from each cell line. To limit ambiguity in cell assignment, only tracks falling within the nuclei segmented region were considered. The kinetic probability distribution clearly distinguishes between the three cell types (Figure 13C). More importantly, by examining the distribution profiles of single-cell states of 103,757 cells from five separate 384-well plates, grouped by their distribution profiles, highly consistent estimates of protein movement at the single-cell level were recovered (Figure 13D). Furthermore, combining data from multiple cells can provide predicted distributions (Figure 13E). Furthermore, combining data from many cells allows for spot-on or state-sequence analysis, with only 10 3 Even trajectories of 10,000 FOV allow for satisfactory inference of the basal state. Imaging for just a few seconds per FOV yields enough trajectories (>10,000) to accurately estimate protein motion, enabling the platform's overall throughput to exceed 90,000 FOV per day, a data acquisition rate that enables drug screening on achievable timescales.

[0225] Equipped with an htSMT system capable of measuring protein motion, the following disclosure demonstrates that measurements of protein motion can be used to functionally characterize proteins. For example, by using steroid hormone receptors (SHRs), which transition between inactive and active states via ligand binding (Figure 14A), the htSMT workflow described herein can capture functionally relevant differences. Specifically, in the absence of hormone, the four SHRs exhibit similar motion profiles, i.e., a small immobile fraction, a large freely moving fraction, and average diffusion coefficients of 3.4–4.3 μm. 2 The fraction bound (f) is shown for each SHR (Figure 14B). No correlation between movement and protein size was observed, highlighting the differences between cellular protein movement and purified systems. Underscoring the selectivity and sensitivity of htSMT, upon addition of agonist, a dramatic increase in immobile tracks due to chromatin binding is observed. As shown in Figure 14B, for each SHR, the fraction bound (f bound ) is 0.1 μm 2 The ER bound to the ER was determined as the fraction of tracks moving at less than 1 / sec. Consistent with previous findings, the proportion of bound molecules was higher in some SHRs than in others, regardless of the presence or absence of ligand. The ligand-induced effect was most pronounced in the ER, which was 34% bound in basal conditions and 87% bound after estradiol treatment (Figure 14B).

[0226] Supporting their use as a representative target family for htSMT analysis, SHRs exhibited high selectivity for their cognate agonists in biochemical binding assays, which was confirmed by measuring dose-dependent kinetic changes as a function of agonist concentration. bound (Fig. 14C) and a decrease in the free diffusion coefficient (D free , Figure 14D) varied among SHRs. Dose-finding curves also showed variable potency (EC50) for each SHR / hormone pair, with ER-estradiol being the most potent and most selective pair. Thus, htSMT screening allows for accurate and precise differentiation of ligand / target specificities directly within a living cellular environment.

[0227] In another example of the htSMT screening assay, next-generation ER degraders, such as GDC-0927, AZD9833, and GDC-9545, were optimized to enhance ER degradation. Indeed, compound-induced changes in protein persistence, e.g., ER degradation, were observed in both established breast cancer model lines and the U2OS expression system (Figures 17A and 22A). Structural analogs of GDC-0927 have been reported and optimized for ER degradation, but the correlation between ER degradation and cell proliferation is poor (Figures 17B and 22B-22D). However, measuring protein persistence provides a more accurate measure of inhibitory activity than can be achieved by assessing protein degradation. The potency and maximum efficacy of structural analogs of GDC-0927 were determined using htSMT. Overall, these analogs exhibited a potency range of 15 pM to 12 nM, with ER f bound increased by 0.4-0.56 (Figure 17C). Small changes in chemical structure resulted in measurable changes in both the potency and maximal efficacy of the compounds as determined using htSMT.

[0228] The potency of GDC-0927 and analogs, determined via either ER degradation or htSMT, was compared to the ability of each of these compounds to block estrogen-induced breast cancer cell proliferation. Potency assessed by ER degradation was not a good predictor of potency in cell proliferation assays (Figure 17C). In contrast, f bound SMT measurements of 1000 cells correlate strongly with cell viability (Figure 17D; R 2 (0.83 for T47d and 0.84 for MCF7). Interestingly, the EC50 values ​​for SMT were on average 10-fold lower than those observed in cell proliferation assays, indicating that SMT is sensitive enough to allow the selection of chemical lines that are ineffective in other cellular assays. Coupled with the throughput of the SMT system, it is possible to detect protein motility (e.g., f boundThis correlation between the effect on protein function (inhibition of cell proliferation) and the activity of the agonist has made it an attractive approach to identify protein modulators with novel properties.

[0229] In addition to known ER activity modulators, many other compounds present in the bioactivity library tested were shown to be effective against f bound To define the threshold for calling a molecule "active" from the screen, bound Ninety-two compounds with different magnitudes of change in f were selected and retested in a dose titration (Figures 23A, 23B). bound A 5% change in β was sufficient to reproducibly distinguish active compounds. Using this approach, 239 compounds affecting ER mobility were identified in the bioactivity library (Figure 15). Among these compounds, correlation between the two screening replicates was high (R 2 = 0.92), and activity levels were reproducible (slope of active molecules was 0.94). While some active compounds could be clustered based on scaffold homology, most clusters consisted of one or a few members (Figures 23C and 23D). Structural clustering was employed to identify known ER modulators for which vendor-provided annotations were not clearly defined. These results demonstrate the reproducibility and robustness of htSMT for screening large collections of molecules.

[0230] Most active molecules obtained in the screen were structurally unrelated to steroids (Figures 23C, 23D). On the other hand, many compounds could be classified based on their reported biological targets or pathways (Figures 18A, 24A, 24B). For example, heat shock protein (HSP) and proteasome inhibitors consistently showed f bound whereas cyclin-dependent kinase (CDK) and mTOR inhibitors increased f boundAlthough many CDK inhibitors lack family specificity (Fig. 24A, pan-CDK), CDK9-specific inhibitors were found to affect ER motility more potently than CDK4 / 6-specific inhibitors. Furthermore, inhibitors targeting ALK, BTK, and FLT3 kinases, which have not been shown to interact with the ER, as well as selective AR and GR antagonists, did not affect ER motility when assessed using SMT (Fig. 18A).

[0231] For the identified inhibitors of cellular pathways, we used dose titration to more fully characterize their respective effects on ER trafficking. Potencies were in the subnanomolar to low-micromolar range (Figure 18B), similar to the reported potencies of these compounds against their cellular targets. Furthermore, these molecules were tested against the AR (Figure 24C) and PR (Figure 24D). Each SHR significantly differed from the other SHRs in their response to compounds identified through ER-focused screening efforts. Again, the extent of ER SMT effects was largely consistent within target classes (Figure 18B, Figures 24A-24D). The finding that structurally distinct compounds exhibited similar effects based on their biological targets supports the notion that these biological targets themselves interact with the ER, thus indirectly affecting ER trafficking. HSP90 is a chaperone for many proteins, including SHR. In the standard model, hormone binding releases the SHR-HSP90 complex. Indeed, HSP90 inhibitors inhibit the activity of ER, AR, and PR. boundThis increased ER activity, consistent with a chaperone function regulating the equilibrium of SHR binding to chromatin (Figures 24B-24D). Proteasome inhibition also resulted in ER immobilization on chromatin, consistent with results obtained in htSMT screening of bioactive compounds. ER has been shown to be phosphorylated by CDK, Src, or GSK-3 via MAPK and PI3K / AKT signaling pathways; therefore, inhibition of these pathways is reasonably expected to affect ER motility measured using SMT. While inhibition of CDK resulted in increased ER mobility, inhibition of PI3K, AKT, or other upstream kinases had no effect (Figure 18A).

[0232] Interestingly, the SMT kinetics of an ER triple point mutant (S104A / S106A / S118A) engineered to lack previously defined phosphorylation sites important for transactivation was affected by CDK and mTOR pathway inhibitors (Figure 25). This suggests that additional phosphorylation sites mediate the effects of CDK9 and PI3K / AKT signaling, or that other molecular targets of CDK and PI3K / AKT can act indirectly to alter ER kinetics. The changes in ER protein kinetics by characterized pathway inhibitors, such as those targeting CDK and mTOR, are subtle but consistent across compounds, demonstrating the biological significance of these observations and highlighting the need for accurate and precise SMT measurements. Thus, the htSMT technique described herein provides a means to provide comprehensive pathway interaction information.

[0233] Furthermore, to support the fact that monitoring changes in binding via changes in target motion can support the identification of pharmacologically relevant compounds, known agonists and antagonists of AR were assayed (Figure 26). AR agonists also showed f bound While AR antagonists increase f both as single treatments and when co-administered with AR agonists, bound This causes a decrease in f bound Increase in f boundBoth the reduction of ATP and the reduction of ATP may be useful to identify mechanistically distinct mechanisms for pharmacological interactions with the fluorescent target protein under observation.

[0234] In addition to changes in kinetics associated with chromatin binding, htSMT can also be used to monitor and identify pharmacological compounds that inhibit or promote protein-protein interactions and protein conformational changes. One such example is the disruption of the ubiquitination process, which relies on a series of protein-protein interactions. Using a known antagonist that disrupts the underlying protein-protein interaction, we show that disruption of the complex results in a significant change in the motion of one of the proteins involved in the interaction (target A), allowing that protein to move more freely (Figure 27). Similarly, htSMT can be used to examine protein conformational changes and screen for compounds that allosterically modulate targets. Known ATP-competitive and allosteric inhibitors of multiple receptor tyrosine kinases, e.g., target B and target C, induce significant changes in protein motion. Furthermore, htSMT is highly sensitive to allosteric inhibition, resulting in kinetic changes that are 4-8 times greater than enzyme inhibitors (Figure 28). Thus, the availability of htSMT allows meaningful investigation of both protein-chromatin and protein-protein interactions.

[0235] 4.2.htSMT binding In certain embodiments of the htSMT workflow described herein, the compound is, in certain cases, f bound Once a compound is identified that increases the static binding of a target (e.g., the static binding of ER to chromatin in the presence of a compound) referred to as increasing k*, a second assay can be performed to obtain more detailed information about the nature of the static binding of the target. For example, the systems and methods described herein can be used to measure an increase in the residence time of the target with respect to a binding partner (i.e., k*). off This can be adapted to distinguish between recovery after exposure to a compound driven by a decrease in ATP (reduction in ATP).

[0236] For example, but not by way of limitation, a bioactivity screen of 5,067 molecules surprisingly found that all known ER modulators (both agonists, such as estradiol, and potent antagonists, such as fulvestrant) exhibited f bound Subsequently, a subset of selective ER modulators (SERMs) and selective ER degraders (SERDs) were evaluated in more detail. All of these molecules competitively bind to the ER ligand-binding domain. As in the bioactivity screen, both SERDs and SERMSs exhibited f bound (Figure 16A) and D free GDC-0927 and GDC-0810 slightly reduced the measured f (Figure 21A). The potencies ranged from 9 pM for GDC-0927 to 4.8 nM for GDC-0810 (Figure 16B). Despite their different physico-chemical properties, all five reduced f within minutes of compound addition. bound ER dissociation from the chaperone complex, dimerization, and chromatin binding appear to occur rapidly and on seemingly comparable time scales. Because the individual steps in these transitions cannot be distinguished, the on-rate of the entire process is estimated by the effective rate constant k* on Importantly, selective AR and GR antagonists did not induce significant modulation of ER motility, further highlighting the utility of the htSMT technique described herein in characterizing the specificity of interactions between modulators of protein function and their cognate targets (Figure 21D).

[0237] Interestingly, the SERMs 4-hydroxytamoxifen (4OHT) and GDC-0810 significantly improved f boundThe maximum increase in f measured by SMT is lower (Figure 16D). A similar effect was previously reported using fluorescence recovery after photobleaching (FRAP), which was confirmed using the Halo-ER cell line (Figure 16E). The delay in ER signal recovery after 2 min in FRAP was consistent with the maximum increase in f measured by SMT. bound These changes were consistent with those of f (Fig. 16F). bound While FRAP has been used to measure differences in ER activity, this technique faces scalability challenges and relies heavily on prior assumptions about fundamental motions within the sample. In contrast, the htSMT approach described herein allows for detailed characterization of the potency of 4OHT and GDC-0810 compared to other ER ligands in their ability to increase ER chromatin binding.

[0238] Both FRAP and htSMT increase the dwell time (k* off decrease in chromatin binding rate (k* on It is not possible to distinguish between recovery caused by bound By modifying the SMT acquisition conditions to reduce the illumination intensity and collect long frame exposures, only immobile proteins form spots. Under these imaging conditions, the distribution of track lengths provides a measure of relative residence times. Both agonist and antagonist treatments result in longer binding times compared to DMSO, indicating that ligand binding increases k* off (Figure 16G, Figure 21E). Consistent with FRAP, estradiol, GDC-0927, and fulvestrant exhibit longer binding times compared to other ER modulators. bound and k* off Using the measured value of k* on In all cases, the change in dissociation rate is f bound and therefore the increase in k* imposed by the ligand onIncreases in ER likely contribute to the observed changes in chromatin-associated ER fraction (Figure 21E). Without being bound by theory, these data are consistent with a model in which ER rapidly binds to chromatin regardless of which molecule occupies the ligand-binding domain, but that some ligands induce conformations on chromatin that can be further stabilized by cofactors. Thus, these data support the idea that ER engages with chromatin through distinct mechanisms. Effective ER inhibitors may promote rapid and transient chromatin binding that does not effectively recruit the cofactors necessary to drive transcription.

[0239] 4.3.Kinetic SMT In certain embodiments of the htSMT workflows described herein, the systems and methods are adapted to determine the rate at which changes in protein motion occur. For example, in certain such embodiments, htSMT can be used to distinguish between direct and indirect effects. Additionally or alternatively, determining the rate at which changes in protein motion occur provides the ability to assess cellular permeability and / or active transport, target efflux and / or influx, target engagement on-rates and / or target engagement off-rates, among other parameters.

[0240] Given the live-cell setup for SMT, a data collection mode (kinetic SMT or kSMT) was configured that allows for measuring protein movement after compound addition at set intervals. Both ER agonists and antagonists rapidly induce ER immobilization on chromatin as measured by kSMT (t in the case of estradiol). 1 / 2 = 1.6 min, Figure 16C). In contrast, HSP90 inhibitors such as ganetespib and HSP990 showed a delay of 5–7 min before changes in ER motility appeared, followed by t = 19.3 and 17.5 min, respectively. 1 / 2 Def boundIncreases in ER activity were observed. The overall effect of these compounds plateaued after 1 hour (Figure 18C). Proteasome inhibitors, such as bortezomib and carfilzomib, acted even more slowly, with changes in ER kinetics only apparent after 40 minutes and slowly increasing over a 4-hour measurement window (Figure 18D). Similarly, differential kinetics of on-target, on-pathway, and off-target inhibitors that alter the kinetics of several additional targets, including target A, target B, and helicase, have been measured (Figure 29). Thus, this exploration of SMT kinetics provides an important tool that can facilitate the distinction between on-target and on-pathway modulators. The kSMT technology described herein enables, for example, rapid mechanistic characterization of active compounds in drug discovery efforts.

[0241] To further distinguish the effects of pathway inhibitors on ER protein movement, we characterized the relative ER residence time of each such molecule. Estradiol, SERMs, and SERDs all increased residence time and thus likely increased the rate of association with ER and chromatin (Figures 16A-16D, Figure 21E). In contrast, HSP90 inhibition by HSP990 and ganetespib significantly increased the rate of ER protein movement. bound Although HSP90 inhibition resulted in an increase in k*, a two-fold and four-fold decrease in the total number of long binding events was observed, respectively, while binding times were similar to those observed with DMSO alone (Figure 18E). These results suggest that HSP90 inhibition significantly increased k* on mainly increases k* off These results demonstrate that ER-chromatin binding can be regulated by altering the rates of association or dissociation, and that inhibition of specific cellular partners can differentially affect these rates. Taken together, the distinct kinetics of direct ER, HSP90, and proteasome modulators suggest that each class of molecule alters ER motility through distinct mechanisms.

[0242] 5. Exemplary Embodiments A. The present disclosure provides a method, the method comprising: receiving an image sequence visualizing the motion of the molecule; Linking molecules between images; generating possible trajectories for each molecule with associated probabilities using a variational Bayesian optimization algorithm and based on linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0243] B. The present disclosure provides a method, the method comprising: receiving an image sequence visualizing the motion of the molecule; Linking molecules between images; generating possible trajectories for each molecule with associated probabilities using a Gibbs sampling algorithm and based on linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0244] C. The present disclosure provides a method, the method comprising: receiving an image sequence visualizing the motion of the molecule; Linking molecules between images; generating possible trajectories for each molecule with associated probabilities using an adaptive hill-climbing algorithm and based on linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0245] C1. The method of any of A-C, wherein at least a subset of the image sequences comprises at least 100 molecules per image.

[0246] C2. The method of any of A-C1, wherein at least a subset of the image sequences comprises at least 1000 molecules per image.

[0247] C3. The method of any of A-C2, wherein at least a subset of the image sequences comprises at least 10,000 molecules per image.

[0248] C4. The method of any of A-C3, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

[0249] C5. The method of any of A-C4, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

[0250] C6. Labeling molecules in a biological sample; causing the biological sample to emit fluorescence; generating an image sequence while the biological sample is fluorescing; The method according to any one of A to C5, further comprising:

[0251] C7. The method of C6, wherein generating the image sequence is performed using a microscope system.

[0252] C8. The method of any one of A-C7, wherein the molecule is imaged in a living cell.

[0253] C9. Inferring a probabilistic dynamic model containing information characterizing the trajectories of molecules;

[0254] The method according to any one of A to C8, further comprising:

[0255] C10. The method of C9, wherein the stochastic dynamic model includes a state array, and the method further includes populating the state array with information characterizing the trajectories of the molecules.

[0256] C11. Further including generating an internal metric of confidence based on the associated probability, wherein the provided data includes the generated internal metric of confidence.

[0257] The method according to any one of A to C10.

[0258] C12. The method of C11, wherein the generated internal confidence metric is a tracking error rate floor that defines a floor for the rate of misconnections made by linking.

[0259] C13. The generated internal metric is Calculating the confidence level of each trajectory;

[0260] The method according to C11, comprising:

[0261] C14. Generating dynamic metrics independent of specific trajectories;

[0262] The method according to any one of A to C13, further comprising:

[0263] C15. The method of any one of A to C14, wherein the linking comprises obtaining data having a plurality of statistical data extracted from the total number of detections or the number of detections in a cell.

[0264] C16. A method according to any of A-C15, wherein providing the data includes one or more of: visualizing at least some of the generated possible trajectories with associated probabilities in a graphical user interface; storing at least some of the generated possible trajectories with associated probabilities in a physical persistent state; loading at least some of the generated possible trajectories with associated probabilities into memory; or transmitting at least some of the generated possible trajectories with associated probabilities to a remote computing device over a network.

[0265] C17. The method of any of A-C16, wherein at least some of the image sequences include consecutive images from a corresponding video.

[0266] C18. The method of any of A-C17, wherein at least some of the image sequences used for linking are non-consecutive images from a corresponding video.

[0267] D. The present disclosure provides a method for single molecule tracking, the method comprising: receiving an image sequence visualizing the movement of the molecule, the image sequence including a first type generated using a first imaging modality and a second type generated using a second, different imaging modality, the method further comprising: Detecting spots in a sequence of images of a first type; linking the detected spots in the sequence of images of the first type to trajectories using a probabilistic tracking algorithm; Segmenting the second type of image sequence to generate a plurality of instance masks; assigning molecules in the second type of image sequence to at least one instance mask of the plurality of instance masks; providing data characterizing the links and allocations to a consuming application or process; Includes:

[0268] D1. The method of D, wherein the probabilistic tracking algorithm comprises a variational Bayesian optimization algorithm.

[0269] D2. The method of D, wherein the probabilistic tracking algorithm includes a Gibbs sampling algorithm.

[0270] D3. The method of D, wherein the partially probabilistic tracking algorithm includes an adaptive hill-climbing algorithm.

[0271] D4. The method according to any one of D to D3, wherein the first imaging modality and the second imaging modality comprise different molecular labeling techniques.

[0272] D5. A method according to any one of D to D4, wherein the first type of image sequence is a single molecule tracking (SMT) video and the second type of image sequence is a non-SMT video.

[0273] D6. The method according to any one of D to D5, wherein the detected spots contain intracellular components.

[0274] D7. The method of any of D-D6, wherein types of molecules in the first type of image sequence are labeled with distinct fluorophores.

[0275] D8. The method of any of D-D7, wherein at least a subset of the image sequences comprises at least 100 molecules per image.

[0276] D9. The method of any of D-D8, wherein at least a subset of the image sequences comprises at least 1000 molecules per image.

[0277] D10. The method of any of D-D9, wherein at least a subset of the image sequences comprises at least 10,000 molecules per image.

[0278] D11. The method of any of D-D10, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

[0279] D12. The method of any one of D-D11, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

[0280] D13. Labeling molecules in biological samples; causing the biological sample to emit fluorescence; generating at least a portion of the image sequence while the biological sample is fluorescing; The method according to any one of D to D12, further comprising:

[0281] D14. The method according to D13, wherein the generation of the image sequence is performed using a microscope system.

[0282] D15. The method according to any one of D to D14, wherein the molecule is imaged in a living cell.

[0283] D16. Inferring a probabilistic dynamic model containing information characterizing the trajectories of molecules; The method according to any one of D to D15, further comprising:

[0284] D17. The method of D16, wherein the stochastic dynamic model includes a state array, and the method further includes populating the state array with information characterizing the trajectories of the molecules.

[0285] D18. The method of any of D-D17, further comprising generating an internal metric of confidence based on the associated probability, wherein the provided data includes the generated internal metric of confidence.

[0286] D19. The method of D18, wherein the generated internal metric of confidence is a tracking error rate lower bound (ERLB) that defines a lower bound on the rate of misconnections made by linking.

[0287] D20. Generated internal metrics are Calculating the confidence level of each trajectory; The method according to D18, comprising:

[0288] D21. Generating dynamic metrics independent of specific trajectories; The method according to any one of D to D20, further comprising:

[0289] D22. The method according to any one of D to D21, wherein the linking comprises obtaining data having a plurality of statistical data extracted from the total number of detections or the number of detections in a cell.

[0290] D23. A method according to any of D-D22, wherein providing the data includes one or more of: visualizing at least some of the generated possible trajectories with associated probabilities in a graphical user interface; storing at least some of the generated possible trajectories with associated probabilities in a physical persistent state; loading at least some of the generated possible trajectories with associated probabilities into memory; or transmitting at least some of the generated possible trajectories with associated probabilities over a network to a remote computing device.

[0291] D24. Generating a plurality of statistical metrics associated with at least one of the trajectories or at least one instance mask; The method according to any one of D to D23, further comprising:

[0292] D25. Remembering the hierarchy of instance masks,

[0293] The method of D24, further comprising:

[0294] D26. A method according to any one of D-D25, wherein at least a portion of the image sequence comprises consecutive images from a corresponding video.

[0295] D27. A method according to any one of D-D26, wherein at least some of the image sequences used for linking are non-consecutive images from corresponding moving images.

[0296] D28.Detecting A method according to any of D-D27, utilizing one or more of a generalized log-likelihood ratio spot detector, a difference of Gaussians (DoG) detector, a Laplacian of Gaussians (LoG) detector, or a Hessian determinant (DoH) blob detector.

[0297] D29. Associating detected spots with spatiotemporal coordinates using sub-pixel localization; The method according to any one of D to D28, further comprising:

[0298] D30.Subpixel Localization The method described in D29, including one or more of a radially symmetric localizer, or maximum likelihood fitting to a candidate spot model using the Levenberg-Marquardt method.

[0299] D31. The method of any of D-D30, wherein the field of view corresponds to at least a portion of a well.

[0300] D32. The image sequence is generated by an apparatus for fluorescence microscopy, the apparatus a light source capable of emitting fluorescent excitation light, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C, and the apparatus further comprising: a first optical element or assembly configured to receive the fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; and a second optical element or assembly including a water immersion objective configured to tilt the light beam with respect to the z-axis in the x-z plane, the second optical element being further configured to focus the light beam onto a sample surface located in the x-y plane, thereby illuminating a portion of the sample surface, and the apparatus further comprising: a detector device configured to receive light from the illuminated portion of the sample surface, the detector device forming one or more projection images based on the light received from the illuminated portion of the sample surface; A method according to any one of A to D31.

[0301] D33. The method of D32, wherein the apparatus includes a second objective lens configured to direct light emitted from the illuminated portion of the sample surface to a detector device.

[0302] D34. The method of D32 or D33, wherein the detector device comprises a semiconductor sensor.

[0303] D35. The method of any of D32-D34, wherein the apparatus includes a third optical element or assembly configured to translate the light beam in the imaging plane in a direction orthogonal to the longer dimension of the light beam.

[0304] D36. The method of D35, wherein the third optical element or assembly includes a galvo mirror.

[0305] D37. The method of any of D32-D36, wherein the detector device includes a semiconductor sensor and the detector device supports a shutter mode for synchronizing translation of the light beam in the sample plane with selective activation or readout of the semiconductor sensor.

[0306] D38. Image sequences are generated by a microscope system for tracking the motion of molecules, the microscope system a stage for supporting a sample, the sample including molecules, and the microscope system further comprising: a light source that emits a light beam capable of inducing a light-based response from molecules in the sample, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C; a water immersion objective for focusing a light beam onto at least a portion of a sample plane, the molecules being disposed at the sample plane, and the microscope system further comprising: a detector device for monitoring a light-based response from the molecule, the response being analyzed to track the movement of the molecule; A method according to any one of A to D31.

[0307] D39. The method of D38, wherein the microscope system further includes a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling an increase in the total field of view of the microscope system in the xy plane.

[0308] D40. The method of D39, wherein the microscope system further comprises a z-position controller of the sample plane, the z-position controller enabling maintenance of focus in the z-direction.

[0309] D41. The method according to any one of D38 to D40, wherein the samples are placed in open wells of a sample plate.

[0310] D42. The method of D41, wherein the sample plate comprises a plurality of open wells.

[0311] D43. The method of D41 or D42, wherein the microscope system further includes an xy position controller for changing a field of view of the microscope system, the changed field of view encompassing a different subset of the plurality of open wells.

[0312] D44. The method of any of D41-D43, wherein the microscope system further includes a temperature-controlled environment configured to control the environment of the sample plate.

[0313] D45. The method of D44, wherein the samples are placed in open wells of a sample plate maintained at 20% to 95% humidity.

[0314] D46. The method of D44 or D45, wherein the samples are placed in open wells of a sample plate maintained at 5% CO2.

[0315] D47. The microscope system further includes an automated sample handling robotic system that allows high-throughput manipulation of multiple samples on a stage, the robotic system comprising: Memory and a processor in communication with the memory; one or more robotic end effectors in communication with the processor, the one or more end effectors manipulating the plurality of specimens on the stage based on communication with the processor; A method according to any one of D38 to D46.

[0316] E. The present disclosure provides a system, the system comprising: at least one data processor; a memory storing instructions that, when executed by at least one data processor, result in operations for implementing the method of any of A-D31; Includes:

[0317] E1. Further comprising an apparatus for fluorescence microscopy, the apparatus comprising: a light source capable of emitting fluorescent excitation light, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C, and the apparatus further comprising: a first optical element or assembly configured to receive the fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; and a second optical element or assembly including a water immersion objective configured to tilt the light beam with respect to the z-axis in the x-z plane, the second optical element being further configured to focus the light beam onto a sample surface located in the x-y plane, thereby illuminating a portion of the sample surface, and the apparatus further comprising: a detector device configured to receive light from the illuminated portion of the sample surface, the detector device forming one or more projection images based on the light received from the illuminated portion of the sample surface; The system described in E.

[0318] E2. The system of E1, wherein the apparatus includes a second objective lens configured to direct light emitted from the illuminated portion of the sample surface to a detector device.

[0319] E3. The system of E1 or E2, wherein the detector device includes a semiconductor sensor.

[0320] E4. A system according to any of E1-E3, wherein the apparatus includes a third optical element or assembly configured to translate the light beam within the imaging plane in a direction perpendicular to the longer dimension of the light beam.

[0321] E5. The system of E4, wherein the third optical element or assembly includes a galvo mirror.

[0322] E6. A system according to any of E1-E5, wherein the detector device includes a semiconductor sensor, and the detector device supports a shutter mode for synchronizing translation of the light beam at the sample plane with selective activation or readout of the semiconductor sensor.

[0323] E7. Further comprising a microscope system for tracking the movement of molecules, the microscope system comprising: a stage for supporting a sample, the sample including molecules, and the microscope system further comprising: a light source that emits a light beam capable of inducing a light-based response from molecules in the sample, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17°C + / - 5°C; a water immersion objective for focusing a light beam onto at least a portion of a sample plane, the molecules being disposed at the sample plane, and the microscope system further comprising: a detector device for monitoring a light-based response from the molecule, the response being analyzed to track the movement of the molecule; The system described in E.

[0324] E8. The system of E7, wherein the microscope system further includes a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction perpendicular to the longer dimension of the light beam, thereby enabling an expansion of the total field of view of the microscope system in the xy plane.

[0325] E9. The system of E8, wherein the microscope system further includes a z-position controller for the sample plane, the z-position controller enabling maintenance of focus in the z-direction.

[0326] E10. The system of any one of E7 to E9, wherein the sample is placed in an open well of a sample plate.

[0327] E11. The system of E10, wherein the sample plate comprises a plurality of open wells.

[0328] E12. The system of E11, wherein the microscope system further comprises an xy position controller for altering a field of view of the microscope system, the altered field of view encompassing a different subset of the plurality of open wells.

[0329] E13. The system of E11 or E12, wherein the microscope system further includes a temperature-controlled environment configured to control the environment of the sample plate.

[0330] E14. The system described in E13, wherein the samples are placed in open wells of a sample plate maintained at 20%-95% humidity.

[0331] E15. The system of E13 or E14, wherein the sample is placed in an open well of a sample plate maintained at 5% CO2.

[0332] E16. The microscope system further includes an automated sample handling robotic system that allows high-throughput manipulation of multiple samples on a stage, the robotic system comprising: a memory for storing instructions; at least one data processor; one or more robotic end effectors in communication with the at least one data processor, the one or more end effectors manipulating the plurality of specimens on the stage based on communication with the at least one data processor; The system according to any one of E7 to E15.

[0333] F. The present disclosure provides a non-transitory computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing device, perform the method described in any of A-D31.

[0334] G. The present disclosure provides a system, the system comprising: means for receiving an image sequence visualizing the molecular motion; a means for linking molecules between images; means for generating possible trajectories for each molecule with associated probabilities based on the linking; a means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0335] H. The present disclosure provides a system, the system comprising: means for receiving an image sequence visualizing the molecular motion; a means for linking molecules between images; means for generating possible trajectories for each molecule with associated probabilities using a variational Bayesian optimization algorithm and based on linking; a means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0336] I. The present disclosure provides a system, the system comprising: means for receiving an image sequence visualizing the molecular motion; a means for linking molecules between images; means for generating possible trajectories for each molecule with associated probabilities using a Gibbs sampling algorithm and based on linking; a means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0337] J. The present disclosure provides a system, the system comprising: means for receiving an image sequence visualizing the motion of the molecule; a means for linking molecules between images; means for generating possible trajectories for each molecule with associated probabilities using an adaptive hill-climbing algorithm and based on linking; a means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Includes:

[0338] K. The present disclosure provides a single molecule tracking system, the system comprising: and means for receiving image sequences visualizing the movement of the molecule, the image sequences including a first type generated using a first imaging modality and a second type generated using a second, different imaging modality, the system further comprising: means for detecting spots in a sequence of images of a first type; means for linking detected spots in the sequence of images of the first type to trajectories using a probabilistic tracking algorithm; means for segmenting the second type of image sequence to generate a plurality of instance masks; means for assigning molecules in a sequence of images of a second type to at least one instance mask of a plurality of instance masks; means for providing data characterizing the links and allocations to a consuming application or process; Includes:

[0339] 6. Working Example Example 1: High-throughput single-molecule tracking of steroid hormone receptors A. Preface Steroid hormone receptors (SHRs) are a class of transcription factors that play critical roles in normal human development and disease pathogenesis. For example, SHRs, such as the estrogen receptor (ESR1 and ESR2), androgen receptor (AR), and progesterone receptor (PR), contribute critically to the acquisition of secondary sexual characteristics, while the glucocorticoid receptor (GR) helps regulate both metabolism and inflammation. In the absence of ligand, SHRs are kept sequestered in multiprotein complexes by the chaperone HSP9021. Typically, in the presence of hormone, they dimerize, bind to their cognate genomic response elements, and recruit epigenetic modifiers and transcriptional machinery. Concurrently, steroid hormone receptor-derived signals impose a significant disease burden by promoting the proliferation of breast (ER) or prostate (AR) cancers or by imposing immune and metabolic dysfunction (GR). Therefore, SHR represents an excellent proof-of-concept system for the study of protein movement as a determinant of protein function, as a wealth of information and reagents are already available for these systems and there are previous reports characterizing several aspects of cell movement.

[0340] This example describes an industrial-scale htSMT technique, a system incorporating such htSMT, hardware and software associated with such htSMT, and methods for using such htSMT. For example, the htSMT technique described herein is capable of measuring protein movement in over 1,000,000 cells per day. Furthermore, using the ER as a proof-of-concept system, the htSMT technique described herein demonstrates specific, robust, and reproducible results. The htSMT technique described herein can be used for a variety of applications, including, but not limited to, traditional drug discovery activities such as screening compound libraries and elucidating SAR. Importantly, the htSMT technique described herein can be used to characterize the contributions of both known and novel pathways to interaction networks, such as protein signaling interaction networks.

[0341] B. Results a. Creation and verification of htSMT system We developed a robotic system capable of handling reagents, collecting high-quality, high-speed SMT image series, and processing the time-ordered raw images to generate molecular trajectories and extract biologically interesting features within defined cellular compartments (Figure 1). To examine the performance of our htSMT system across a wide range of diffusion coefficients, we generated three U2OS cell lines expressing HaloTag fusion proteins with well-established intracellular behavior. Histone H2B-Halo is primarily incorporated into chromatin and therefore largely immobile over short periods of time, and was used to estimate localization error. A prenylation motif embedded in the plasma membrane (Halo-CaaX) exhibits moderate diffusion. Unfused HaloTag was chosen to represent the upper limit of "free" diffusion in cells. Single-molecule trajectories measured in these cell lines yielded the expected diffusion coefficients (Figure 13A). Using immobile H2B-Halo trajectories, the localization error of our htSMT system was found to be 39 nm (Figure 19A), which is comparable to other benchmark strobe illumination datasets.

[0342] We tested whether the htSMT platform could be scaled up to extract accurate molecular trajectories. Using 384-well plates, we mixed free Halo, Halo-CaaX, and H2B-Halo cell lines in equal proportions in each well. Imaging with a 94 μm × 94 μm field of view (FOV) achieved an average of 10 nuclei simultaneously (Figure 13B, Figure 20B), sufficient for most FOVs to contain cells from each cell line. To limit ambiguity in cell assignment, we only considered tracks that fell within the nuclear segmented region. The probability distributions of diffusion states clearly distinguished the three cell types (Figure 13C). More importantly, examining the distribution profiles of single-cell states of 103,757 cells from five separate 384-well plates, grouped by their distribution profiles, yielded highly consistent estimates of protein motion at the single-cell level (Figure 13D).

[0343] While single-cell measurements are powerful, the number of trajectories in a single cell is limited, which can lead to broad estimates of the diffusion state. However, combining trajectories from multiple cells yields the expected distribution of diffusion states (Figure 13E). Furthermore, combining trajectories from many cells enables spot-on or state-sequence analysis, allowing satisfactory inference of the basal diffusion state with as few as 10 trajectories. Imaging for just a few seconds per FOV yields enough trajectories (>10,000) to accurately estimate protein motion, enabling the overall throughput of the platform to reach 13,000 wells (>90,000 FOVs per day, >1,000,000 cells per day). This is a data acquisition rate that enables drug screening on feasible timescales (Figure 30).

[0344] b. Measurement of protein movement in SHR using htSMT Armed with the htSMT system, which can measure protein movement over a wide range, the following work demonstrates that measurements of protein movement can be used to characterize protein activity. SHRs transition between inactive and active states upon ligand binding (Figure 14A), and htSMT can capture these differences. To this end, we prepared HaloTag-fused ER, AR, PR, and GR cell lines in the U2OS cell background to minimize the impact of comparing movement across different cell types. Clones were carefully selected to ensure that HaloTag-fused SHRs were comparable in transcript abundance to each other and not higher than transcript levels in tissue-specific cell lines such as MCF7 and T47d, which are both ER- and PR-positive (Figure 31).

[0345] In the absence of hormone, all four proteins exhibited similar kinetic profiles, i.e., small immobile and large freely diffusing fractions, with average diffusion coefficients of 3.4–4.3 μm 2The fraction of immobile tracks due to chromatin binding is shown (Fig. 14B). No correlation between diffusion and protein size was observed, highlighting the differences between cellular protein movement and purified systems. Upon addition of agonist, a dramatic increase in immobile tracks due to chromatin binding is observed. For each SHR, the fraction bound (f bound ) is 0.1 μm 2 The ER bound to the ER was determined as the fraction of tracks diffusing at less than 1 / sec. Consistent with previous findings, some SHRs had a higher proportion of bound molecules than others, regardless of the presence or absence of ligand. The ligand-induced effect was most pronounced in the ER, where 34% bound in basal conditions and 87% bound after estradiol treatment (Figure 14B).

[0346] SHRs exhibit high selectivity for their cognate agonists in biochemical binding assays, which was confirmed by measuring dose-dependent changes in locomotor activity as a function of agonist concentration. 結合 (Fig. 14C) and a decrease in the free diffusion coefficient (D free , Figure 14D) varied among SHRs. Dose-titration curves also demonstrated variable potency (EC50) for each SHR / hormone pair, with ER-estradiol being the most potent and most selective pair. RNA-seq after estradiol stimulation demonstrated significant induction of a distinct set of ER-dependent genes, confirming that the increased chromatin binding observed by SMT has the functional effect of promoting ER-responsive gene programs, even in an ectopic expression setting (Figures 32 and 33). Thus, SMT can accurately and precisely distinguish ligand / target specificities directly within a living cellular environment.

[0347] c. Screening a diverse set of bioactive chemicals identifies known and novel modulators of ER dynamics Collectively, our attempts to characterize ligand selectivity for AR, ER, GR, and PR suggested that SMT could be used to investigate the effects of compounds on protein dynamics with a throughput that facilitates high-throughput screening. Next, we investigated the specificity and sensitivity of the htSMT platform. We screened a structurally diverse set of 5,067 molecules with heterogeneous biological activity against the ER and compared the f bound The changes in ER activity were assessed (Figure 15). The screen was performed twice to assess reproducibility, showing high concordance between replicates for ER-active molecules (Figure 15). Each compound measurement was averaged from SMT trajectories between 94 and 161 cells (25th-75th percentiles, Figure 20A). This screen demonstrates that the htSMT platform described herein has significant advantages over more manual, low-throughput methods.

[0348] For each plate, the assay window of the screen was robust (Figure 20B, average Z' factor = 0.79). In each case, the measured potency of the control estradiol remained within 3-fold of the mean (Figures 20C and 20D), and the distribution of negative control wells was centered around zero (Figure 20E). Of the 30 compounds identified from the bioactive set expected to modulate the ER as either agonists or antagonists, all exhibited f as measured by SMT, including prominent examples such as 4-hydroxytamoxifen, fulvestrant, and bazedoxifene. boundを The results showed a significant increase (Fig. 15 and Fig. 35).

[0349] A somewhat counterintuitive finding has been reported for the ER: that both strong agonists and antagonists can lead to increased chromatin binding, but this does not appear to be a general feature of SHRs. While the PR antagonist mifepristone exhibits effects similar to those of ER antagonists (Figures 36A-36B), AR antagonists such as enzalutamide and darolutamide, as well as GR antagonists such as AL082D06, cause a decrease in chromatin binding. This occurs when administered alone or when coadministered in competition with their cognate agonists (Figures 36C-36D). These results demonstrate how important the cellular context and interaction partners are to understanding the effects of compounds on their intended targets. To emphasize this point, we identified numerous active compounds targeting diverse nodes of the ER interaction network, including modulators of the proteasome, chaperones, and kinases, in addition to binders of the ER ligand-binding domain (Figure 18A).

[0350] d. Cellular ER dynamics explain structure-activity relationships (SAR) of ER modulators In a bioactivity screen of 5,067 molecules, it was surprising to find that all known ER modulators (both agonists, such as estradiol, and potent antagonists, such as fulvestrant) exhibited f bound Subsequently, a subset of selective ER modulators (SERMs) and selective ER degraders (SERDs) were evaluated in more detail. All of these molecules competitively bind to the ER ligand-binding domain. As in the bioactivity screen, both SERDs and SERMSs exhibited f bound (Figure 16A) and D free GDC-0927 and GDC-0810 slightly reduced the measured f (Figure 21A). The potencies ranged from 9 pM for GDC-0927 to 4.8 nM for GDC-0810 (Figure 16B). Despite their different physico-chemical properties, all five reduced f within minutes of compound addition. boundER dissociation from the chaperone complex, dimerization, and chromatin binding appear to occur rapidly and on seemingly comparable time scales. Because the individual steps in these transitions cannot be distinguished, the on-rate of the entire process is estimated by the effective rate constant k* on Importantly, selective AR and GR antagonists did not induce significant modulation of ER motility, further highlighting the utility of the htSMT technique described herein in characterizing the specificity of interactions between modulators of protein function and their cognate targets (Figure 21D).

[0351] Interestingly, the SERMs 4-hydroxytamoxifen (4OHT) and GDC-0810 significantly improved f bound The maximum increase in f measured by SMT is lower (Figure 16D). A similar effect was previously reported using fluorescence recovery after photobleaching (FRAP), which was confirmed using the Halo-ER cell line (Figure 16E). The delay in ER signal recovery after 2 min in FRAP was consistent with the maximum increase in f measured by SMT. bound These changes were consistent with those of f (Fig. 16F). bound While FRAP has been used to measure differences in ER activity, this technique faces scalability challenges and relies heavily on prior assumptions about fundamental motions within the sample. In contrast, the htSMT approach described herein allows for detailed characterization of the potency of 4OHT and GDC-0810 compared to other ER ligands in their ability to increase ER chromatin binding.

[0352] Both FRAP and htSMT increase the dwell time (k* off decrease in chromatin binding rate (k* on It is not possible to distinguish between recovery caused by boundBy modifying the SMT acquisition conditions to reduce the illumination intensity and collect long frame exposures, only immobile proteins form spots. Under these imaging conditions, the distribution of track lengths provides a measure of relative residence times. Both agonist and antagonist treatments result in longer binding times compared to DMSO, indicating that ligand binding increases k* off (Figure 16G, Figure 21E). Consistent with FRAP, estradiol, GDC-0927, and fulvestrant exhibit longer binding times compared to other ER modulators. bound and k* off Using the measured value of k* on In all cases, the change in dissociation rate is f bound and therefore the increase in k* imposed by the ligand on Increases in ER likely contribute to the observed changes in chromatin-associated ER fraction (Figure 21E). Without being bound by theory, these data are consistent with a model in which ER rapidly binds to chromatin regardless of which molecule occupies the ligand-binding domain, but that some ligands induce conformations on chromatin that can be further stabilized by cofactors. Thus, these data support the idea that ER engages with chromatin through distinct mechanisms. Effective ER inhibitors may promote rapid and transient chromatin binding that does not effectively recruit the cofactors necessary to drive transcription.

[0353] e.htSMT can define relevant structure-activity relationships of ER antagonists As their names suggest, next-generation ER degraders, such as GDC-0927, AZD9833, and GDC-9545, were optimized to enhance ER degradation. Indeed, compound-induced ER degradation by immunofluorescence was observed in both established breast cancer model lines and the U2OS ectopic expression system (Figures 17A and 22A). Structural analogs of GDC-0927 have been reported and optimized for ER degradation, but the correlation between ER degradation and cell proliferation is poor (Figures 17B and 22B-22D). However, by measuring protein motion, a more accurate measure of inhibitory activity can be obtained than can be achieved by assessing protein degradation. The potency and maximum efficacy of structural analogs of GDC-0927 were determined using htSMT. Overall, these analogs exhibited a potency range of 15 pM to 12 nM, with ER f bound increased by 0.4-0.56 (Figure 17C). Small changes in chemical structure resulted in measurable changes in both the potency and maximal efficacy of the compounds as determined using SMT.

[0354] The potency of GDC-0927 and analogs, determined via either ER degradation or SMT, was compared to the ability of each of these compounds to block estrogen-induced breast cancer cell proliferation. Potency assessed by ER degradation was not a good predictor of potency in cell proliferation assays (Figure 17C). In contrast, f bound SMT measurements of 1000 cells correlate strongly with cell viability (Figure 16E; R 2 (0.83 for T47d and 0.84 for MCF7). Interestingly, the EC50 values ​​for SMT were on average 10-fold lower than those observed in cell proliferation assays, indicating that SMT is sensitive enough to allow the selection of chemical lines that are ineffective in other cellular assays. Coupled with the throughput of the SMT system, it is possible to detect protein motility (e.g., f bound This correlation between the effect on protein function (inhibition of cell proliferation) and the activity of the agonist has made it an attractive approach to identify protein modulators with novel properties.

[0355] f. Individual pathway interactors exhibit unique phenotypes related to their impact on ER motility In addition to known ER activity modulators, many other compounds in our bioactivity library have been shown to bound To define the threshold for calling a molecule "active" from the screen, bound Ninety-two compounds with different magnitudes of change in f were selected and retested in a dose titration (Figures 23A, 23B). bound A 5% change in β was sufficient to reproducibly distinguish active compounds. Using this approach, 239 compounds affecting ER mobility were identified in the bioactivity library (Figure 15). Among these compounds, correlation between the two screening replicates was high (R 2 = 0.92), and activity levels were reproducible (slope of active molecules was 0.94). While some active compounds could be clustered based on scaffold homology, most clusters consisted of one or a few members (Figures 23C and 23D). To identify known ER modulators for which vendor-provided annotations were not clearly defined, we employed structural clustering (Figure 22C). These results demonstrate the reproducibility and robustness of htSMT for screening large collections of molecules.

[0356] Most active molecules obtained in the screen were structurally unrelated to steroids (Figures 23C, 23D). On the other hand, many compounds could be classified based on their reported biological targets or pathways (Figures 18A, 24A, 24B). For example, heat shock protein (HSP) and proteasome inhibitors consistently showed f bound whereas cyclin-dependent kinase (CDK) and mTOR inhibitors increased f boundAlthough many CDK inhibitors lack family specificity (Fig. 24A, pan-CDK), CDK9-specific inhibitors were found to affect ER motility more potently than CDK4 / 6-specific inhibitors. Furthermore, inhibitors targeting ALK, BTK, and FLT3 kinases, which have not been shown to interact with the ER, as well as selective AR and GR antagonists, did not affect ER motility when assessed using SMT (Fig. 18A).

[0357] For the identified inhibitors of cellular pathways, we used dose titration to more fully characterize their respective effects on ER trafficking. Potencies were in the subnanomolar to low-micromolar range (Figure 18B), similar to the reported potencies of these compounds against their cellular targets. Furthermore, these molecules were tested against the AR (Figure 24C) and PR (Figure 24D). Each SHR significantly differed from the other SHRs in their response to compounds identified through ER-focused screening efforts. Again, the extent of ER SMT effects was largely consistent within target classes (Figure 18B, Figures 24A-24D). The finding that structurally distinct compounds exhibited similar effects based on their biological targets supports the notion that these biological targets themselves interact with the ER, thus indirectly affecting ER trafficking. HSP90 is a chaperone for many proteins, including SHR. In the standard model, hormone binding releases the SHR-HSP90 complex. Indeed, HSP90 inhibitors inhibit the activity of ER, AR, and PR. boundThis increased ER activity, consistent with a chaperone function regulating the equilibrium of SHR binding to chromatin (Figures 24B-24D). Proteasome inhibition also resulted in ER immobilization on chromatin, consistent with results obtained in htSMT screening of bioactive compounds. ER has been shown to be phosphorylated by CDK, Src, or GSK-3 via MAPK and PI3K / AKT signaling pathways; therefore, inhibition of these pathways is reasonably expected to affect ER motility measured using SMT. While inhibition of CDK resulted in increased ER mobility, inhibition of PI3K, AKT, or other upstream kinases had no effect (Figure 18A).

[0358] Interestingly, the SMT kinetics of an ER triple point mutant (S104A / S106A / S118A) engineered to lack previously defined phosphorylation sites important for transactivation was affected by CDK and mTOR pathway inhibitors (Figure 25). This suggests that additional phosphorylation sites mediate the effects of CDK9 and PI3K / AKT signaling, or that other molecular targets of CDK and PI3K / AKT can act indirectly to alter ER kinetics. The changes in ER protein kinetics by characterized pathway inhibitors, such as those targeting CDK and mTOR, are subtle but consistent across compounds, demonstrating the biological significance of these observations and highlighting the need for accurate and precise SMT measurements. Thus, the htSMT technique described herein provides a means to provide comprehensive pathway interaction information.

[0359] Because SMT can identify compounds that act either directly on the target or through some intermediate process, strategies to distinguish between these alternative modes of action were sought. For example, by investigating the rate at which changes in protein movement appear, SMT can be used to distinguish between direct and indirect effects on ER activity. Given the live-cell setup of SMT, a data collection mode (kinetic SMT or kSMT) was configured that allows protein movement after compound addition to be measured at set intervals. Both ER agonists and antagonists rapidly induce ER anchoring on chromatin (t in the case of estradiol) as measured by kSMT. 1 / 2 = 1.6 min, Figure 16C). In contrast, HSP90 inhibitors such as ganetespib and HSP990 showed a delay of 5–7 min before changes in ER motility appeared, followed by t = 19.3 and 17.5 min, respectively. 1 / 2 Def bound An increase in ER activity was observed. The overall effect of these compounds plateaued after 1 hour (Figure 18C). Proteasome inhibitors, such as bortezomib and carfilzomib, acted more slowly, with changes in ER kinetics appearing only after 40 minutes and gradually increasing over a 4-hour measurement window (Figure 18D). Therefore, exploring the kinetics of SMT provides an important tool that can facilitate the distinction between on-target and on-pathway modulators. The kSMT technology described herein enables rapid mechanistic characterization of active compounds, for example, in drug discovery efforts.

[0360] To further distinguish the effects of pathway inhibitors on ER protein movement, we characterized the relative ER residence time of each such molecule. Estradiol, SERMs, and SERDs all increased residence time and thus likely increased the rate of association with ER and chromatin (Figures 15, 21E). In contrast, HSP90 inhibition by HSP990 and ganetespib significantly increased the rate of ER protein movement. boundAlthough HSP90 inhibition resulted in an increase in k*, a two-fold and four-fold decrease in the total number of long binding events was observed, respectively, while binding times were similar to those observed with DMSO alone (Figure 18E). These results suggest that HSP90 inhibition significantly increased k* on mainly increases k* off These results demonstrate that ER-chromatin binding can be regulated by altering the rates of association or dissociation, and that inhibition of specific cellular partners can differentially affect these rates. Taken together, the distinct kinetics of direct ER, HSP90, and proteasome modulators suggest that each class of molecule alters ER motility through distinct mechanisms.

[0361] C. Method a.Cell line U2OS (ATCC Catalog No. HTB-96), MCF7 (ATCC Catalog No. HTB-22), T47d (ATCC Catalog No. HTB-133), and SK-BR-3 (ATCC Catalog No. HTB-30) cells were grown in DMEM (Cat. No. 1056601, Gibco DMEM, High Glucose, GlutaMAX Supplement, Thermofisher) supplemented with 10% fetal bovine serum (Cat. No. 16000044, Thermofisher) and 1% penicillin-strep (Cat. No. 15140122, Thermofisher), maintained in a humidified 37°C incubator with 5% CO2, and subcultured approximately every 2–3 days.

[0362] b. HaloTag-expressing cell line For ER, AR, and PR-HaloTag fusions, mammalian expression vectors containing the fusion genes under the control of the weak L30 promoter and a neomycin resistance marker were transfected into U2OS cells at 70% confluence using FuGENE6 (Cat. No. E2691, Promega). Transfected cells were selected with 500 μg / mL G418 (Cat. No. 10131027, Thermo Fisher Scientific) and then clonally isolated. Clones expressing the desired fusion genes were first isolated by centrifugation with 100 nM JF. 549 -HTL (Cat. No. GA1110, Promega) and 50 nM Hoechst 33342 staining, JF 549 The expected distribution of signal was determined by identifying clones. Three to six clones were then tested for response to control compounds using SMT conditions, and the most homogeneous clones were subsequently expanded for further testing. Unless otherwise specified, all experiments use a single clonally isolated cell line. Because U2OS cells endogenously express GR, a HaloTag was inserted immediately before the endogenous NR3C1 stop codon via homologous recombination repair using CRISPR / Cas9. HTL-JF 646 HaloTag knock-in was verified by staining imaging and through DNA sequencing.

[0363] C. Western blot Cells were grown under the same conditions as described above. 1.5 x 106 cells were seeded per well in DMEM medium in a 6-well plate and cultured overnight, followed by compound treatment (DMSO or 100 nM fulvestrant) for 24 hours the following day. Cells were lysed in 200 μL of 1X Cell Lysis Buffer (Cat. No. 9803, Cell Signaling). Protein lysate concentrations were then determined using a BCA Protein Assay Kit (Cat. No. 23225, Pierce™ BCA Protein Assay Kit) according to the manufacturer's instructions. Capillary Western immunoassays were then performed using Jess Protein Simple according to the manufacturer's instructions (Protein Simple, USA). αER (1:100, RM-9101) levels were normalized to the loading control β-tubulin (1:100, NC0244815LI-COR92642213, Thermo Fisher Scientific). Peaks were analyzed with Compass software (Proteinimple, USA).

[0364] d.RNA-seq Cells were seeded into 12-well tissue culture-treated plates at a density of 250,000 (U2OS-WT), 200,000 (U2OS-ER), or 300,000 (MCF7, SK-BR-3, T47d) cells per well. After 24 hours, cells were treated with estradiol at a final concentration of 25 nM at the indicated time points (0, 10, 60, or 3 hours). To process cells for total RNA, cells were washed twice with ice-cold PBS, lysed in 350 μL of Buffer RLT (Qiagen 79216), detached from plates (Fisher 08100241), frozen on dry ice, and stored at -20°C. Cell lysates were then thawed, homogenized using a QIAshredder column (Qiagen 79656), and processed with a Qiagen RNeasy Micro kit (Qiagen 74004) using standard protocols, including an optional on-column DNase digestion step (Qiagen 79254). All samples had a RIN score of 10 using a TapeStation (Agilent 5067-5576). RNA sequencing libraries were prepared from total RNA by Novogene (CA). Briefly, mRNA was purified from total RNA and fragmented using poly-T oligo-conjugated magnetic beads. First-strand synthesis was performed using random hexamer primers, and second-strand synthesis was performed using dTTP. Libraries were prepared after end repair, A-tailing, adapter ligation, amplification, and purification. Libraries were sequenced on an Illumina NovaSeq with paired 150-cycle reads. For data analysis, Hisat2 v2.0.5 was used to align paired-end reads to the hg38 reference genome, featureCounts v1.5.0-p3 was used to count the number of reads mapped to each gene, and DESeq2 (1.20.0) was used to perform differential expression analysis.

[0365] e. Single molecule tracking sample preparation Cells were seeded at 6000 cells per well in tissue-culture-treated 384-well glass-bottom plates. The seeded cells were then incubated overnight at 37°C and 5% CO2 to allow attachment. For all SMT experiments, cells were incubated with 5–100 pM JF. 549 The cells were incubated with β-HTL (catalog no. GA1110, Promega) and 50 nM Hoechst 33342 in complete medium for 1 hour. Afterwards, cells were washed three times with DPBS and twice with imaging medium. The imaging medium was fluorBrite DMEM medium (catalog no. A1896701, Thermo Fisher Scientific) supplemented with GlutaMAX (catalog no. 35050079, Thermo Fisher Scientific) and the same serum and antibiotics as the growth medium. Where appropriate, compounds were serially diluted in Echo-certified 384-well low-dead-volume source microplates (0018544, Beckman Coulter) to generate dose-titrated source material. Compounds were administered in cell culture medium at a final dilution of 1:1000. Each compound dose had at least two replicates per plate and three plate replicates. 20 DMSO control wells and two no-dye control wells were randomly assigned to each plate. Unless otherwise specified, compounds were incubated at 37°C for 1 h before image acquisition.

[0366] f. Image acquisition Unless otherwise noted, all image acquisition using SMT was performed on a custom-built HILO microscope based on a Nikon Ti2 microscope, a motorized stage, an upper-stage environmental chamber (OKO Labs), a 4-band filter cube (Chroma), and custom laser emitters with wavelengths of 405 nm and 561 nm, delivering >10 mW and >150 mW of power, respectively, at the back focal plane of the objective. Fluorescence emission was passed through a high-speed filter wheel (Finger Lakes Instruments) and collected with a backlit CMOS camera (Prime95b, Teledyne). Images were acquired with a 60x 1.27NA water-immersion objective (Nikon). The environmental chamber was set at 37°C, 95% humidity, and 5% CO2. For each field of view, 200 SMT frames were collected at a frame rate of 100 Hz using a 2-ms strobe laser pulse. For downstream registration of tracks to the nucleus, 10 frames of the Hoechst channel were collected at the same frame rate.

[0367] g. Image analysis Image acquisition yields one JF per field of view 549 A video and one Hoechst were generated. JF 549 The video is from individual JF 549 Hoechst videos were used to track molecular movement, and Hoechst videos were used for nuclear segmentation. Tracking was achieved in three sequential steps: detection, subpixel localization, and linking, using a combination of existing methods. Briefly, spots were detected using a generalized log-likelihood ratio detector. After detection, starting from an initial guess obtained by a radial symmetry method, Levenberg-Marquardt fitting with a unified 2D Gaussian spot model was used to refine the estimated location of each emitter to subpixel resolution. Detected spots were linked into trajectories using a custom modification of a hill-climbing algorithm. The same detection, subpixel localization, and linking setup was used for all videos used in this manuscript.

[0368] For nuclei segmentation, all frames of the Hoechst movie were averaged to generate an average projection. This average projection was then segmented using a neural network trained on manually labeled nuclei. Each spot was assigned to at most one nucleus using its subpixel coordinates.

[0369] To recover motion information from the trajectories, we used a Bayesian inference approach, state array, with the "RBME" likelihood function and a time scale ranging from 0.01 to 100.0 μm. 2 s -1 A grid of 100 diffusion coefficients and 31 localization error sizes ranging from 0.02 to 0.08 μm was used. After inference, the localization error was minimized to obtain a one-dimensional distribution across the diffusion coefficients for each field of view. For single-cell analysis, SMT and nuclear segmentation were performed on a mixture of U2OS cells carrying H2B-HaloTag, HaloTag-CaaX, or free HaloTag. The marginal likelihood of each set of 100 diffusion coefficients for the set of trajectories within each segmented nucleus was then evaluated. These marginal likelihood functions were clustered using k-means (3 clusters), and the marginal likelihood functions for each cell were ordered by their cluster index to generate a heatmap. The fractional boundary values ​​(f bound ) to estimate 0.1 μm 2 s -1 The posterior distribution of the state sequence less than σ is integrated. free ) to estimate 0.1 μm 2 s -1 We calculated the mean of the posterior distribution over

[0370] h. Data Analysis Tracking results from the automated processing pipeline were analyzed using KNIME or Spotfire (TIBCO). bound or D free The measured values ​​of f were linked to the experimental metadata and aggregated for each condition. bound The change in f of each well bound f of DMSO in the same plate boundThe EC50 was calculated as the difference from the median of the DMSO wells. Wells that contained no cells within the field of view or where the field of view was out of focus were excluded from further analysis. Compounds were evaluated for assay interference using the median fluorescence intensity of the tracking channel and were excluded if it was more than three standard deviations higher than the median intensity of the DMSO wells. Similarly, plates that failed to clearly separate the active and negative controls or whose performance deviated significantly from the rest of the screen were excluded from further analysis. Finally, compounds with a variance greater than three standard deviations above the mean compound variance (41 compounds, 0.08%) were excluded from downstream analysis. The Z' factor between the active control and DMSO on a plate was calculated as previously described. EC50 values ​​were calculated in Prism (GraphPad) by first log-transforming the molecule concentrations and then fitting them to a four-parameter logistic curve.

[0371] i. Clustering of active molecules Chemical structure-based clustering was performed on the molecules identified as active (239 in total). Molecular frameworks were calculated as described by Murcko et al. and implemented in Pipeline Pilot. The molecular frameworks were clustered using functional class fingerprinting (FCFP_4) with a similarity threshold cutoff of 0.3 Tanimoto distance. A total of 21 clusters were obtained, of which singletons were the dominant class (124 molecules). The next most abundant group was the flavone class, represented by 27 members, followed by several distinct classes within the steroid class, each consisting of 14 and 20 members. The other category was the stilbene class, which included seven members, one of which represented tamoxifen. The remaining active substances (47 molecules) were grouped into one three-member cluster, and all others were grouped into two members per cluster.

[0372] j.Kinetic Experiments Cells were seeded into 384-well plates the day before and stained and washed as described above. One well with 25 FOVs per well was taken as a baseline reading. Compounds were then manually added to each well to a final concentration of 100 nM during imaging. Data were then collected for 20 wells. Pauses were included between each FOV to ensure the entire imaging plan covered the assay window. bound The change in t was determined for each well relative to t=0.

[0373] For the 4-hour assay, plates were imaged twice using 8 FOVs at different FOV positions per well to prevent photobleaching from affecting the data. All data presented were run in three different biological replicates.

[0374] k. Dwell time imaging Sample preparation and execution of dwell-time imaging experiments were performed in a similar manner to the single-molecule tracking assay described above, with a few exceptions. Samples were prepared using 1–10 pM JF. 549 The cells were stained with 50 nM Hoechst 33342 (Promega) for 1 hour. The camera integration time was set to 250 ms, and the laser power at the objective was reduced to 5 mW, collecting 400 frames per field of view. The laser was continuously on during image acquisition. Compound incubations ranged from 1 to 4 hours. At least 8 replicate wells were collected per condition.

[0375] l.Residence time analysis Image processing, including spot detection, localization, and track reconnection, was performed using the same methods as described above. For dwell-time imaging, individual localizations were limited to a maximum displacement of 300 nm for individual jump reconnections to selectively track slow-diffusing molecules. The set of trajectories for each field of view was binned into a 1-CDF distribution and fitted to a two-exponential decay model: CDF(t) = A(Fe-kfastt + (1-F)e-kslowt)CDFt = A(Fe-kfastt + 1-Fe-kslowt).

[0376] m. Fluorescence recovery after photobleaching Images were acquired with a custom-built HiLo microscope using a Spectra Light Engine RS-232, as described above. Stimulation was performed directly using a miniscanner coupled with a Coherent OBIS 561 nm 100 mW laser. All imaging was performed using a 60x 1.27 NA water immersion objective (Nikon). All experiments were performed at 37°C. For FRAP experiments, cells were seeded in 384-well plates the day before and treated with 50 nM HTL-JF. 549 The cells were labeled with β-glucan and washed as described above. Compounds were added to a final concentration of 100 nM before imaging. Pre-bleaching images were then obtained by averaging 10 consecutive images. Eight to ten regions were bleached (two background, six to eight cells), and two regions within the cells were left unbleached. The bleached regions were then bleached at 10% power without scanning. Images were acquired every 200 ms for the next 30 s, then every 1 s for 2 min. Background-subtracted mean intensity was measured over time in the region of interest and normalized to the mean fluorescence in the baseline image, then normalized to the unbleached region to account for readout-induced photobleaching of the fluorophore. Data from 18 to 24 cells per experiment were pooled across three biological experiments.

[0377] Immunofluorescence Cells were grown under the conditions described above. Cells were seeded at 6,000 cells per well for Halo-ER U2OS cells and 8,000 cells per well for MCF7 and T47d cells in glass-bottom 384-well plates coated with 0.05 mg / ml PDL (catalog no. A3890401, Thermofisher). Cells were cultured overnight and treated with compounds for 24 hours on day 2 at 37°C and 5% CO2. Compounds were serially diluted in Echo® Certified 384-well low-dead-volume source microplates (0018544, Beckman Coulter) to generate a 21-point dose response at 1:3 dilutions, starting at 10 mM. Compounds were administered in cell culture medium at a final dilution of 1:1000. Dose responses of 8 to 12 points were selected based on the potency of each compound. Each concentration was replicated at least once per plate, resulting in at least two plate replicates. Cells were fixed with 4% paraformaldehyde (catalog no. 15710-S, Electron Microscopy Sciences) for 20 minutes. They were then permeabilized for 1 hour at room temperature using a blocking buffer containing 1% bovine serum albumin and 0.3% Triton-X100 in 1x PBS. Immunofluorescence staining of ER was performed using αER antibody (1:500, RM-9101) diluted in the same blocking buffer for 1 hour at room temperature. Extensive washing with PBS was performed before secondary antibody staining. Secondary antibody staining was performed using Alexa Fluor 488-conjugated anti-rabbit IgG (1:1000, catalog no. A32731, Thermos Fisher) for 1 hour. Nuclear staining was performed using a 1 mg / ml Hoechst 33342 solution. Immunofluorescence imaging was performed using an ImageXpress Micro (Molecular Devices) at 10x magnification with four fields per well. Fluorescence intensity within the nucleus was quantified using CellProfiler. All analyses and curve fitting were performed using Prism with DMSO as the baseline.

[0378] o. Cell proliferation Cells were grown and seeded under the conditions described above. Cells were seeded in 384-well plates (Corning, Catalog No. 353963) at 1,000 cells per well for Halo-ER U2OS, 1,200 cells per well for SK-BR-3, and 1,800 cells per well for MCF7 and T47d. Cells were cultured overnight and then treated with compounds the following day. Compound concentrations and administration were the same as those described above for the immunofluorescence assay. Plates were scanned using phase contrast in an IncuCyte live cell analysis system (Sartorius) at 24-hour intervals for a total of 5 days. Cell proliferation was quantified using the built-in analysis function with a whole-well confluency mask. All analyses and curve fitting were performed using Prism with DMSO as the baseline.

[0379] Example 2: htSMT analysis of AR agonists and antagonists In this example, utilizing the methods described in Example 1, including the preparation and analysis of cell lines expressing AR as a fluorescent target protein, we provide additional evidence that changes in protein interactions, such as changes in protein binding measured via changes in target motion, can support the identification of pharmacologically relevant compounds. Specifically, known agonists and antagonists of AR were assayed as described in Example 1, but the f measured for AR was significantly higher. bound were performed in the presence of 25 nM agonist, 10 μM potent antagonist, or 25 nM and 10 μM combination of agonist and antagonist, respectively.

[0380] AR agonists are f bound It was observed that AR antagonists increased f when administered alone or in combination with AR agonists. bound It was observed that the f bound and f bound This clearly demonstrates that both the reduction of α and β may be useful in identifying mechanistically distinct mechanisms of pharmacological interactions with the fluorescent target protein under observation.

[0381] Example 3: htSMT analysis of target A antagonists In this example, we utilize the methods described in Example 1 for the preparation and analysis of cell lines expressing target A as a fluorescent target protein, providing additional evidence that alterations in protein interactions, e.g., alterations in protein-protein interactions in signaling pathways independent of ER signaling as described in Example 1, can support the identification of pharmacologically relevant compounds. Specifically, Figure 27 shows the changes in the kinetics of target A in response to dose titration of a known target A antagonist. Each series of shapes represents a different compound, and error bars represent SEM. As evidenced by the shape of the curves, increasing concentrations of antagonist induce measurable differences in the median third quartile (Q3) jump length compared to DMSO, indicating release of target A from the bound state in the presence of the antagonist.

[0382] Example 4: htSMT analysis of competitive and allosteric antagonists In this example, we provide further evidence that the preparation and analysis of cell lines expressing exemplary receptor tyrosine kinases (Target B and Target C) and helicase as fluorescent target proteins, for example, compounds that affect protein interactions (e.g., alter protein-protein interactions in signal transduction pathways) via competitive or allosteric inhibition, can support the identification of pharmacologically relevant compounds. Specifically, Figure 28 shows the changes in the kinetics of Target B and Target C in the presence of competitive or allosteric antagonists. Each series of curves represents a different compound normalized to DMSO, and error bars represent SEM. As evidenced by the shape of the curves, increasing concentrations of antagonist induce measurable differences in the median third quartile (Q3) jump length compared to DMSO, indicating that Target B and Target C are differentially affected by competitive and allosteric antagonists.

[0383] Similarly, Figure 29 shows the change in median Q3 jump length relative to DMSO as a function of time after compound addition for target A, target B, and the helicase target. Evidence from the presented data suggests that treating proteins with on-target inhibitors either increases or decreases protein motion. For target A and target B, these changes occur within minutes of compound addition. In contrast, helicases treated with either pathway antagonists or off-target modulators, as in the case of DNA damage induction, require hours or longer for changes in protein motion to reach their maximum effect.

[0384] The subject matter described herein may be embodied in systems, devices, methods, and / or articles, depending on the desired configuration. The embodiments set forth in the foregoing description do not represent all embodiments consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. For example, the above-described embodiments may be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several additional features disclosed above. Furthermore, the logic flow depicted in the accompanying figures and / or described herein does not necessarily require the particular order shown or sequential order to achieve desirable results. Other embodiments may also be within the scope of the following claims.

Claims

1. receiving an image sequence visualizing the motion of the molecule; linking molecules between said images; generating possible trajectories for each molecule with associated probabilities using a variational Bayesian optimization algorithm and based on said linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; A method comprising:

2. receiving an image sequence visualizing the motion of the molecule; linking molecules between said images; generating possible trajectories for each molecule with associated probabilities using a Gibbs sampling algorithm and based on said linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; A method comprising:

3. receiving an image sequence visualizing the motion of the molecule; linking molecules between said images; generating possible trajectories for each molecule with associated probabilities using an adaptive hill-climbing algorithm and based on said linking; providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; A method comprising:

4. 10. A method according to any preceding claim, wherein at least a subset of the image sequence comprises at least 100 molecules per image.

5. 10. A method according to any preceding claim, wherein at least a subset of the image sequence comprises at least 1000 molecules per image.

6. 10. A method according to any preceding claim, wherein at least a subset of the image sequence comprises at least 10,000 molecules per image.

7. 10. A method according to any preceding claim, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

8. 10. A method according to any preceding claim, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

9. labeling a molecule in a biological sample; causing the biological sample to emit fluorescence; generating the image sequence while the biological sample is fluorescing; 10. The method of any preceding claim, further comprising:

10. The method of claim 9 , wherein the generating of the image sequence is performed using a microscope system.

11. 10. The method of any preceding claim, wherein the molecules are imaged in living cells.

12. inferring a probabilistic dynamic model that includes information characterizing the trajectory of the molecule; 10. The method of any preceding claim, further comprising:

13. The method of claim 12 , wherein the stochastic dynamic model comprises a state array, the method further comprising populating the state array with the information characterizing the trajectory of the molecule.

14. 10. The method of any preceding claim, further comprising generating an internal metric of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metric of confidence.

15. The method of claim 14 , wherein the generated internal metric of confidence is a tracking error rate floor that defines a floor for the rate of misconnections made by the linking.

16. The generated internal metric is: Calculating the confidence level of each trajectory; 15. The method of claim 14, comprising:

17. generating dynamic metrics independent of specific trajectories; 10. The method of any preceding claim, further comprising:

18. 10. The method of any preceding claim, wherein said linking comprises obtaining data comprising a plurality of statistical data extracted from the total number of detections or the number of detections in a cell.

19. 10. The method of any preceding claim, wherein providing the data comprises one or more of: visualizing at least some of the generated possible trajectories with associated probabilities in a graphical user interface; storing at least some of the generated possible trajectories with associated probabilities in a physical persistent state; loading at least some of the generated possible trajectories with associated probabilities into a memory; or transmitting at least some of the generated possible trajectories with associated probabilities over a network to a remote computing device.

20. 10. A method according to any preceding claim, wherein at least part of the image sequence comprises successive images from a corresponding motion picture.

21. 10. A method according to any preceding claim, wherein at least some of the image sequences used for said linking are non-sequential images from a corresponding motion picture.

22. 1. A method for single molecule tracking, comprising: receiving an image sequence visualizing molecular motion, the image sequence including a first type generated using a first imaging modality and a second type generated using a second, different imaging modality, the method further comprising: detecting spots in the sequence of images of the first type; linking detected spots in the sequence of images of the first type to trajectories using a probabilistic tracking algorithm; segmenting the sequence of images of the second type to generate a plurality of instance masks; assigning molecules in the image sequence of the second type to at least one instance mask of the plurality of instance masks; providing data characterizing the link and the allocation to a consuming application or process; The method comprising:

23. The method of claim 22 , wherein the probabilistic tracking algorithm comprises a variational Bayesian optimization algorithm.

24. The method of claim 22 , wherein the probabilistic tracking algorithm comprises a Gibbs sampling algorithm.

25. The method of claim 22 , wherein the partially probabilistic tracking algorithm comprises an adaptive hill climbing algorithm.

26. The method of any of claims 22 to 25, wherein the first imaging modality and the second imaging modality comprise different molecular labeling techniques.

27. The method of any of claims 22 to 26, wherein the image sequences of the first type are single molecule tracking (SMT) movies and the image sequences of the second type are non-SMT movies.

28. The method according to any one of claims 22 to 27, wherein the detected spots contain intracellular components.

29. The method of any of claims 22 to 28, wherein types of molecules in the image sequence of the first type are labeled with distinct fluorophores.

30. A method according to any of claims 22 to 29, wherein at least a subset of the image sequence comprises at least 100 molecules per image.

31. A method according to any of claims 22 to 30, wherein at least a subset of the image sequence comprises at least 1000 molecules per image.

32. A method according to any one of claims 22 to 31, wherein at least a subset of the image sequence comprises at least 10,000 molecules per image.

33. A method according to any one of claims 22 to 32, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

34. A method according to any one of claims 22 to 33, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

35. labeling a molecule in a biological sample; causing the biological sample to emit fluorescence; generating at least a portion of the sequence of images while the biological sample is fluorescing; The method of any of claims 22 to 34, further comprising:

36. 36. The method of claim 35, wherein the generating of the image sequence is performed using a microscope system.

37. The method of any of claims 22 to 36, wherein the molecules are imaged in living cells.

38. inferring a probabilistic dynamic model that includes information characterizing the trajectory of the molecule; The method of any of claims 22 to 37, further comprising:

39. 39. The method of claim 38, wherein the probabilistic dynamic model comprises a state array, the method further comprising populating the state array with the information characterizing the trajectory of the molecule.

40. 40. The method of any of claims 22 to 39, further comprising generating an internal metric of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metric of confidence.

41. 41. The method of claim 40, wherein the generated internal metric of confidence is a tracking error rate lower bound (ERLB) that defines a lower bound on the rate of misconnections made by the linking.

42. The generated internal metric is: Calculating the confidence level of each trajectory; 41. The method of claim 40, comprising:

43. generating dynamic metrics independent of specific trajectories; The method of any of claims 22 to 42, further comprising:

44. The method of any of claims 22 to 43, wherein said linking comprises obtaining data having a plurality of statistical data extracted from the total number of detections or the number of detections in a cell.

45. 45. The method of any of claims 22 to 44, wherein providing the data comprises one or more of: visualizing at least some of the generated possible trajectories with associated probabilities in a graphical user interface; storing at least some of the generated possible trajectories with associated probabilities in a physical persistent state; loading at least some of the generated possible trajectories with associated probabilities into a memory; or transmitting at least some of the generated possible trajectories with associated probabilities over a network to a remote computing device.

46. generating a plurality of statistical metrics associated with at least one of the trajectories or the at least one instance mask; The method of any of claims 22 to 45, further comprising:

47. Storing a hierarchy of instance masks, 47. The method of claim 46, further comprising:

48. A method according to any of claims 22 to 47, wherein at least part of the image sequence comprises successive images from a corresponding motion picture.

49. A method according to any of claims 22 to 48, wherein at least some of the image sequences used for said linking are non-sequential images from a corresponding motion picture.

50. The detecting step includes:

50. The method of any of claims 22 to 49, utilizing one or more of a generalized log-likelihood ratio spot detector, a difference of Gaussians (DoG) detector, a Laplacian of Gaussians (LoG) detector, or a determinant of Hessian (DoH) blob detector.

51. associating said detected spots with spatiotemporal coordinates using sub-pixel localization; The method of any of claims 22 to 50, further comprising:

52. The sub-pixel localization may include:

52. The method of claim 51, comprising one or more of a radially symmetric localizer or maximum likelihood fitting to a candidate spot model using the Levenberg-Marquardt method.

53. The method of any of claims 22 to 52, wherein the field of view corresponds to at least a portion of a well.

54. The image sequence is generated by an apparatus for fluorescence microscopy, the apparatus comprising: a light source capable of emitting fluorescence excitation light, said light source exhibiting an output drift of less than about 10% at an ambient temperature of 17° C.+ / −5° C.; a first optical element or assembly configured to receive a fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; a second optical element or assembly including a water immersion objective configured to tilt the light beam with respect to the z-axis in an x-z ​​plane, the second optical element further configured to focus the light beam onto a sample surface located in the x-y plane, thereby illuminating a portion of the sample surface, the apparatus further comprising: a detector device configured to receive light from the illuminated portion of the sample surface, the detector device forming one or more projection images based on the light received from the illuminated portion of the sample surface. A method according to any preceding claim.

55. 55. The method of claim 54, wherein the apparatus includes a second objective lens configured to direct the light emitted from the illuminated portion of the sample surface toward the detector device.

56. 56. The method of claim 54 or 55, wherein the detector device comprises a semiconductor sensor.

57. 57. A method according to any of claims 54 to 56, wherein the apparatus includes a third optical element or assembly configured to translate the light beam in the imaging plane in a direction perpendicular to the longer dimension of the light beam.

58. 58. The method of claim 57, wherein the third optical element or assembly includes a galvo mirror.

59. 59. The method of any of claims 54 to 58, wherein the detector device includes a solid-state sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with selective activation or readout of the solid-state sensor.

60. The image sequence is generated by a microscope system for tracking the movement of the molecule, the microscope system comprising: a stage for supporting a sample, the sample including the molecule, the microscope system further comprising: a light source that emits a light beam capable of eliciting a light-based response from the molecules in the sample, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17° C.+ / −5° C.; and the microscope system further comprising: a water immersion objective for focusing the light beam onto at least a portion of the sample plane, the molecules being disposed at the sample plane, the microscope system further comprising: a detector device for monitoring the light-based response from the molecule, the response being analyzed to track the movement of the molecule; 54. The method of any one of claims 1 to 53.

61. 61. The method of claim 60, wherein the microscope system further comprises a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling an increase in the total field of view of the microscope system in the xy plane.

62. 62. The method of claim 61, wherein the microscope system further comprises a z-position controller for the sample plane, the z-position controller enabling maintenance of focus in the z direction.

63. The method of any of claims 60 to 62, wherein the samples are placed in open wells of a sample plate.

64. 64. The method of claim 63, wherein the sample plate comprises a plurality of open wells.

65. 65. The method of claim 63 or claim 64, wherein the microscope system further includes an xy position controller for changing a field of view of the microscope system, the changed field of view encompassing a different subset of the plurality of open wells.

66. The method of any of claims 63 to 65, wherein the microscope system further comprises a temperature controlled environment configured to control the environment of the sample plate.

67. 67. The method of claim 66, wherein the samples are placed in open wells of the sample plate maintained at a humidity of between 20% and 95%.

68. The samples were incubated in 5% CO ２ 68. The method of claim 66 or claim 67, wherein the sample is placed in an open well of the sample plate maintained at 100° C.

69. The microscope system further includes an automated sample handling robotic system that enables high-throughput manipulation of multiple samples on the stage, the robotic system comprising: Memory and a processor in communication with the memory; one or more robotic end effectors in communication with the processor, the one or more end effectors manipulating the plurality of specimens on the stage based on communication with the processor.

69. The method of any one of claims 60 to 68.

70. at least one data processor; a memory storing instructions which, when executed by said at least one data processor, result in operations for performing a method according to any of claims 1 to 53; Including, the system.

71. and further comprising an apparatus for fluorescence microscopy, said apparatus comprising: a light source capable of emitting fluorescence excitation light, said light source exhibiting an output drift of less than about 10% at an ambient temperature of 17° C.+ / −5° C.; a first optical element or assembly configured to receive a fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; a second optical element or assembly including a water immersion objective configured to tilt the light beam with respect to the z-axis in an x-z ​​plane, the second optical element further configured to focus the light beam onto a sample surface located in an x-y plane, thereby illuminating a portion of the sample surface, the apparatus further comprising: a detector device configured to receive light from the illuminated portion of the sample surface, the detector device forming one or more projection images based on the light received from the illuminated portion of the sample surface.

71. The system of claim 70.

72. 72. The system of claim 71, wherein the apparatus includes a second objective lens configured to direct the light emitted from the illuminated portion of the sample surface toward the detector device.

73. 73. The system of claim 71 or claim 72, wherein the detector device comprises a semiconductor sensor.

74. 74. A system according to any one of claims 71 to 73, wherein the apparatus includes a third optical element or assembly configured to translate the light beam in the imaging plane in a direction perpendicular to the longer dimension of the light beam.

75. 75. The system of claim 74, wherein the third optical element or assembly includes a galvo mirror.

76. 76. A system according to any one of claims 71 to 75, wherein the detector device includes a solid-state sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with selective activation or readout of the solid-state sensor.

77. and a microscope system for tracking the movement of the molecule, the microscope system comprising: a stage for supporting a sample, the sample including the molecule, the microscope system further comprising: a light source that emits a light beam capable of eliciting a light-based response from the molecules in the sample, the light source exhibiting an output drift of less than about 10% at an ambient temperature of 17° C.+ / −5° C.; and the microscope system further comprising: a water immersion objective for focusing the light beam onto at least a portion of the sample plane, the molecules being disposed at the sample plane, the microscope system further comprising: a detector device for monitoring the light-based response from the molecule, the response being analyzed to track the movement of the molecule; 71. The system of claim 70.

78. 78. The system of claim 77, wherein the microscope system further includes a scanning optical element or assembly configured to translate the light beam within the sample plane in a direction perpendicular to the longer dimension of the light beam, thereby enabling an increase in the total field of view of the microscope system in the xy plane.

79. 79. The system of claim 78, wherein the microscope system further includes a z-position controller for the sample plane, the z-position controller enabling maintenance of focus in the z direction.

80. 80. The system of any of claims 77 to 79, wherein the samples are placed in open wells of a sample plate.

81. 81. The system of claim 80, wherein the sample plate comprises a plurality of open wells.

82. 82. The system of claim 81, wherein the microscope system further comprises an xy position controller for changing a field of view of the microscope system, the changed field of view encompassing a different subset of the plurality of open wells.

83. The system of any of claims 81 to 82, wherein the microscope system further comprises a temperature controlled environment configured to control the environment of the sample plate.

84. 84. The system of claim 83, wherein the samples are placed in open wells of the sample plate maintained at a humidity of between 20% and 95%.

85. The samples were incubated in 5% CO ２ 85. The system of claim 83 or claim 84, wherein the sample is positioned within an open well of the sample plate maintained at .

86. The microscope system further includes an automated sample handling robotic system that enables high-throughput manipulation of multiple samples on the stage, the robotic system comprising: said memory for storing instructions; the at least one data processor; and one or more robotic end effectors in communication with the at least one data processor, the one or more end effectors manipulating the plurality of specimens on the stage based on communication with the at least one data processor. A system according to any one of claims 77 to 85.

87. A non-transitory computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing device, perform the method according to any of claims 1 to 53.

88. means for receiving an image sequence visualizing the motion of the molecule; means for linking molecules between said images; means for generating possible trajectories for each molecule with associated probabilities based on said linking; means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Including, the system.

89. means for receiving an image sequence visualizing the motion of the molecule; means for linking molecules between said images; means for generating possible trajectories for each molecule with associated probabilities using a variational Bayesian optimization algorithm and based on said linking; means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Including, the system.

90. means for receiving an image sequence visualizing the motion of the molecule; means for linking molecules between said images; means for generating possible trajectories for each molecule with associated probabilities using a Gibbs sampling algorithm and based on said linking; means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Including, the system.

91. means for receiving an image sequence visualizing the motion of the molecule; means for linking molecules between said images; means for generating possible trajectories for each molecule with associated probabilities using an adaptive hill-climbing algorithm and based on said linking; means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process; Including, the system.

92. 1. A single molecule tracking system, comprising: means for receiving image sequences visualizing molecular motion, the image sequences including a first type generated using a first imaging modality and a second type generated using a second, different imaging modality, the system further comprising: means for detecting spots in said sequence of images of said first type; means for linking detected spots in said sequence of images of said first type to trajectories using a probabilistic tracking algorithm; means for segmenting the sequence of images of the second type to generate a plurality of instance masks; means for assigning molecules in the sequence of images of the second type to at least one instance mask of the plurality of instance masks; means for providing data characterizing said links and said allocations to a consuming application or process; The system comprising:

Images