Systems and methods for high throughput single molecule tracking in living cells

Abstract

Systems and methods for high throughput single molecule tracking in living cells. A sequence of images visualizing movement of molecules is received. Molecules across the images are linked. Using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities are generated. Data characterizing the generated possible trajectories with associated probabilities is provided to a consuming application or process.

Description

Systems and methods for high-throughput single-molecule tracking within living cells receive image sequences visualizing molecular motion. Molecules across the images are linked. A variational Bayesian optimization algorithm is used, based on the links, to generate possible trajectories and their associated probabilities for each molecule. Data characterizing the generated possible trajectories and their associated probabilities is provided to a consumer application or process. (Summary Figure)

Claims

EIK0002 14711-023-228 (101551.228023) What is claimed is:

1. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

2. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using a Gibbs sampling algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

3. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using an adaptive hill climbing algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and 122 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

4. The method of any of the preceding claims, wherein at least a subset of the sequence of images comprise at least 100 molecules per image.

5. The method of any of the preceding claims, wherein at least a subset of the sequence of images comprise at least 1000 molecules per image.

6. The method of any of the preceding claims, wherein at least a subset of the sequence of images comprise at least 10,000 molecules per image.

7. The method of any of the preceding claims, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

8. The method of any of the preceding claims, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

9. The method of any of the preceding claims further comprising: labeling molecules within a biological sample; fluorescing the biological sample; and generating the sequence of images while fluorescing the biological sample. 123 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 10. The method of claim 9, wherein the generating of the sequence of images is performed using a microscopy system.

11. The method of any of the preceding claims, wherein the molecules are imaged within living cells.

12. The method of any of the preceding claims further comprising: inferring a probabilistic dynamical model comprising information characterizing the trajectories of the molecules.

13. The method of claim 12, wherein the probabilistic dynamical model comprises a state array and the method further comprises: populating the state array with the information characterizing the trajectories of the molecules.

14. The method of any of the preceding claims further comprising: generating internal metrics of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metrics of confidence.

15. The method of claim 14, wherein the generated internal metrics of confidence is a tracking error rate lower bound that defines a lower bound on a rate of misconnections made by the linking.

16. The method of claim 14, wherein the generated internal metrics comprise: calculating a confidence level for each trajectory. 124 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 17. The method of any of the preceding claims further comprising: generating dynamical metrics independently of specific trajectories.

18. The method of any of the preceding claims, wherein the linking comprises retrieving data comprising a plurality of statistics extracted from a total number of detections or a number of detections in a cell.

19. The method of any of the preceding claims, wherein the providing of data comprises one or more of: visualizing at least a portion of the generated possible trajectories with associated probabilities in a graphical user interface, storing at least a portion of the generated possible trajectories with associated probabilities in physical persistence, loading at least a portion of the generated possible trajectories with associated probabilities in memory, or transmitting at least a portion of the generated possible trajectories with associated probabilities over a network to a remote computing device.

20. The method of any of the preceding claims, wherein at least a portion of the sequence of images comprise contiguous images from a corresponding movie.

21. The method of any of the preceding claims, wherein at least a portion of the sequence of images used by the linking are non-contiguous images from a corresponding movie. 125 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 22. A method for single molecule tracking comprising: receiving a sequence of images visualizing movement of molecules, the sequence of images comprising a first type generated using a first imaging modality and a second type generated using a second, different imaging modality; detecting spots within the first type of the sequence of images; linking detected spots within the first type of the sequence of images into trajectories using a probabilistic tracking algorithm; segmenting the second type of the sequence of images to generate a plurality of instance masks; assigning molecules within the second type of the sequence of images to at least one instance mask of the plurality of instance masks; and providing data characterizing the linking and assigning to a consuming application or process.

23. The method of claim 22, wherein the probabilistic tracking algorithm comprises a variational Bayesian optimization algorithm.

24. The method of claim 22, wherein the probabilistic tracking algorithm comprises a Gibbs sampling algorithm.

25. The method of claim 22, wherein the partial probabilistic tracking algorithm comprises an adaptive hill climbing algorithm. 126 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 26. The method of any of claims 22 to 25, wherein the first imaging modality and the second imaging modality comprise different molecular labeling techniques.

27. The method of any of claims 22 to 26, wherein the first type of the sequence of images are single molecule tracking (SMT) movies and the second type of the sequence of images are non- SMT movies.

28. The method of any of claims 22 to 27, wherein the detected spots comprise sub-cellular components.

29. The method of any of claims 22 to 28, wherein types of molecules within the first type of the sequence of images are labeled with distinct fluorophores.

30. The method of any of claims 22 to 29, wherein at least a subset of the sequence of images comprise at least 100 molecules per image.

31. The method of any of claims 22 to 30, wherein at least a subset of the sequence of images comprise at least 1000 molecules per image.

32. The method of any of claims 22 to 31, wherein at least a subset of the sequence of images comprise at least 10,000 molecules per image. 127 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 33. The method of any of claims 22 to 32, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

34. The method of any of claims 22 to 33, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

35. The method of any of claims 22 to 34 further comprising: labeling molecules within a biological sample; fluorescing the biological sample; and generating at least a portion of the sequence of images while fluorescing the biological sample.

36. The method of claim 35, wherein the generating of the sequence of images is performed using a microscopy system.

37. The method of any of claims 22 to 36, wherein the molecules are imaged within living cells.

38. The method of any of claims 22 to 37 further comprising: inferring a probabilistic dynamical model comprising information characterizing the trajectories of the molecules. 128 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 39. The method of claim 38, wherein the probabilistic dynamical model comprises a state array and the method further comprises: populating the state array with the information characterizing the trajectories of the molecules.

40. The method of any of claims 22 to 39 further comprising: generating internal metrics of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metrics of confidence.

41. The method of claim 40, wherein the generated internal metrics of confidence is a tracking error rate lower bound (ERLB) that defines a lower bound on a rate of misconnections made by the linking.

42. The method of claim 40, wherein the generated internal metrics comprise: calculating a confidence level for each trajectory.

43. The method of any of claims 22 to 42 further comprising: generating dynamical metrics independently of specific trajectories.

44. The method of any of claims 22 to 43, wherein the linking comprises retrieving data comprising a plurality of statistics extracted from a total number of detections or a number of detections in a cell. 129 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 45. The method of any of claims 22 to 44, wherein the providing of data comprises one or more of: visualizing at least a portion of the generated possible trajectories with associated probabilities in a graphical user interface, storing at least a portion of the generated possible trajectories with associated probabilities in physical persistence, loading at least a portion of the generated possible trajectories with associated probabilities in memory, or transmitting at least a portion of the generated possible trajectories with associated probabilities over a network to a remote computing device.

46. The method of any of claims 22 to 45, further comprising: generating a plurality of statistical metrics associated with at least one of the trajectories or the at least one instance mask.

47. The method of claim 46 further comprising: storing a hierarchy of instance masks.

48. The method of any of claims 22 to 47, wherein at least a portion of the sequence of images comprise contiguous images from a corresponding movie.

49. The method of any of claims 22 to 48, wherein at least a portion of the sequence of images used by the linking are non-contiguous images from a corresponding movie.

50. The method of any of claims 22 to 49, wherein the detecting utilizes one or more of: 130 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) a generalized log likelihood ratio spot detector, a difference-of-Gaussians (DoG) detector, a Laplacian-of-Gaussian (LoG) detector, or a determinant of Hessian (DoH) blob detector.

51. The method of any of claims 22 to 50 further comprising: associating the detected spots with spatiotemporal coordinates using subpixel localization.

52. The method of claim 51, wherein the subpixel localization comprises one or more of: a radial symmetry localizer or a maximum likelihood fit to a candidate spot model using the Levenberg-Marquardt method.

53. The method of any of claims 22 to 52, wherein the field of view corresponds to at least a portion of a well.

54. The method of any of the preceding claims, wherein the sequence of images are generated by an apparatus for fluorescence microscopy, the apparatus comprising: a light source capable of emitting fluorescence excitation light, wherein the light source exhibits power output drift of less than about 10% at an ambient temperature of 17° C + / - 5° C; a first optical element or assembly configured to receive a fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; a second optical element or assembly comprising a water immersion objective configured to incline the light beam relative to the z-axis in an x-z plane, wherein the second optical element is further configured to focus the light beam at a sample plane located in the x-y plane, thereby illuminating at least a portion of the sample plane; and 131 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) a detector device configured to receive light from the illuminated portion of the sample plane, wherein the detector device forms one or more projected images based on the light received from the illuminated portion of the sample plane.

55. The method of claim 54, wherein the apparatus comprises a second objective configured to direct the light emitted from the illuminated portion of the sample plane to the detector device.

56. The method of claim 54 or 55, wherein the detector device comprises a semiconductor sensor.

57. The method of any of claims 54 to 56, wherein the apparatus comprises a third optical element or assembly configured to translate the light beam in the imaging plane in a direction orthogonal to the longer dimension of the light beam 58. The method of claim 57, wherein the third optical element or assembly comprises a galvo mirror.

59. The method of any of claims 54 to 58, wherein the detector device comprises a semiconductor sensor, wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor. 132 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 60. The method of any of claims 1 to 53, wherein the sequence of images are generated by a microscopy system for tracking the movement of a molecule, the microscopy system comprising: a stage for supporting a sample, wherein the sample contains the molecule; a light source for emitting a light beam capable of inducing a light-based response from the molecule in the sample, wherein the light source exhibits power output drift of less than about 10% at an ambient temperature of 17° C + / - 5° C; a water immersion objective for focusing the light beam on at least a portion of the sample plane, wherein the molecule is disposed in the sample plane; and a detector device for monitoring the light-based response from the molecule, which is analyzed to thereby track the movement of the molecule.

61. The method of claim 60, wherein the microscopy system further comprises a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling a larger total field of view of the microscopy system in the x-y plane.

62. The method of claim 61, wherein the microscopy system further comprises a z-position controller for the sample plane, wherein the z-position controller enables maintenance of focus in the z-direction.

63. The method of any of claims 60 to 62, wherein the sample is disposed within an open well of a sample plate. 133 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 64. The method of any of claim 63, wherein the sample plate comprises a plurality of open wells.

65. The method of claim 63 or claim 64, wherein the microscopy system further comprises an x-y position controller for altering a field of view of the microscopy system, the altered fields of view encompassing different subsets of the plurality of open wells.

66. The method of any of claims 63 to 65, wherein the microscopy system further comprises a temperature-controlled environment configured to control the environment of the sample plate.

67. The method of claim 66, wherein the sample disposed within an open well of the sample plate is maintained at 20%-95% humidity.

68. The method of claim 66 or claim 67, wherein the sample disposed within an open well of the sample plate is maintained at 5% CO2.

69. The method of any of claims 60 to 68, wherein the microscopy system further comprises an automated sample-handling robotic system to enable high throughput manipulation of a plurality of samples on the stage, wherein the robotic system comprises: a memory; a processor in communication with the memory; and 134 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) one or more robotic end-effectors in communication with the processor, wherein the one or more end-effectors manipulate the plurality of samples on the stage based on communication with the processor.

70. A system comprising: at least one data processor; and memory storing instructions, which when executed by at least one data processor, result in operations for implementing a method as in any of claims 1 to 53.

71. The system of claim 70, further comprising: an apparatus for fluorescence microscopy having: a light source capable of emitting fluorescence excitation light, wherein the light source exhibits power output drift of less than about 10% at an ambient temperature of 17° C + / - 5° C; a first optical element or assembly configured to receive a fluorescence excitation light source and shape the fluorescence excitation light source to form a light beam; a second optical element or assembly comprising a water immersion objective configured to incline the light beam relative to the z-axis in an x-z plane, wherein the second optical element is further configured to focus the light beam at a sample plane located in the x-y plane, thereby illuminating at least a portion of the sample plane; and a detector device configured to receive light from the illuminated portion of the sample plane, wherein the detector device forms one or more projected images based on the light received from the illuminated portion of the sample plane. 135 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 72. The system of claim 71, wherein the apparatus comprises a second objective configured to direct the light emitted from the illuminated portion of the sample plane to the detector device.

73. The system of claim 71 or claim 72, wherein the detector device comprises a semiconductor sensor.

74. The system of any of claims 71 to 73, wherein the apparatus comprises a third optical element or assembly configured to translate the light beam in the imaging plane in a direction orthogonal to the longer dimension of the light beam 75. The system of claim 74, wherein the third optical element or assembly comprises a galvo mirror.

76. The system of any of claims 71 to 75, wherein the detector device comprises a semiconductor sensor, wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

77. The system of claim 70, further comprising: a microscopy system for tracking the movement of a molecule having: a stage for supporting a sample, wherein the sample contains the molecule; 136 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) a light source for emitting a light beam capable of inducing a light-based response from the molecule in the sample, wherein the light source exhibits power output drift of less than about 10% at an ambient temperature of 17° C + / - 5° C; a water immersion objective for focusing the light beam on at least a portion of the sample plane, wherein the molecule is disposed in the sample plane; and a detector device for monitoring the light-based response from the molecule, which is analyzed to thereby track the movement of the molecule.

78. The system of claim 77, wherein the microscopy system further comprises a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling a larger total field of view of the microscopy system in the x-y plane.

79. The system of claim 78, wherein the microscopy system further comprises a z-position controller for the sample plane, wherein the z-position controller enables maintenance of focus in the z-direction.

80. The system of any of claims 77 to 79, wherein the sample is disposed within an open well of a sample plate.

81. The system of claim 80, wherein the sample plate comprises a plurality of open wells. 137 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 82. The system of claim 81, wherein the microscopy system further comprises an x-y position controller for altering a field of view of the microscopy system, the altered fields of view encompassing different subsets of the plurality of open wells.

83. The system of any of claims 81 to 82, wherein the microscopy system further comprises a temperature-controlled environment configured to control the environment of the sample plate.

84. The system of claim 83, wherein the sample disposed within an open well of the sample plate is maintained at 20%-95% humidity.

85. The system of claim 83 or claim 84, wherein the sample disposed within an open well of the sample plate is maintained at 5% CO2.

86. The system of any of claims 77 to 85, wherein the microscopy system further comprises an automated sample-handling robotic system to enable high throughput manipulation of a plurality of samples on the stage, wherein the robotic system comprises: the memory storing instructions; the at least one data processor; and one or more robotic end-effectors in communication with the at least one data processor, wherein the one or more end-effectors manipulate the plurality of samples on the stage based on communication with the at least one data processor. 138 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) 87. A non-transitory computer program product storing instructions, which when executed by at least one data processor forming part of at least one computing device, implement a method as in any of claims 1 to claim 53.

88. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

89. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

90. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; 139 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) means for generating, using a Gibbs sampling algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

91. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, using an adaptive hill climbing algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

92. A single molecule tracking system comprising: means for receiving a sequence of images visualizing movement of molecules, the sequence of images comprising a first type generated using a first imaging modality and a second type generated using a second, different imaging modality; means for detecting spots within the first type of the sequence of images; means for linking detected spots within the first type of the sequence of images into trajectories using a probabilistic tracking algorithm; means for segmenting the second type of the sequence of images to generate a plurality of instance masks; 140 NAI-1538902512v1EIK0002 14711-023-228 (101551.228023) means for assigning molecules within the second type of the sequence of images to at least one instance mask of the plurality of instance masks; and means for providing data characterizing the linking and assigning to a consuming application or process. 141 NAI-1538902512v1

Images