Systems and methods for high throughput single molecule tracking in living cells

Abstract

Systems and methods for high throughput single molecule tracking in living cells. A sequence of images visualizing movement of molecules is received. Molecules across the images are linked. Using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities are generated. Data characterizing the generated possible trajectories with associated probabilities is provided to a consuming application or process.

Description

11(d) Translation of Abstract A system and method for high-throughput single-molecule tracking within living cells receives a sequence of images visualizing molecular motion. Molecules across the images are linked. A variational Bayesian optimization algorithm is used, based on the links, to generate possible trajectories for each molecule and their associated probabilities. Data characterizing the generated possible trajectories and their associated probabilities is provided to a consumer application or process. Abstract

Claims

EIK0007 14711-024-228 (101551.228024) What is claimed is:

1. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

2. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using a Gibbs sampling algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

3. A method comprising: receiving a sequence of images visualizing movement of molecules; linking molecules across the images; generating, using an adaptive hill climbing algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and 167 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

4. The method of any of the preceding claims, wherein at least a subset of the sequence of images comprise at least 100 molecules per image.

5. The method of any of the preceding claims, herein at least a subset of the sequence of images comprise at least 1000 molecules per image.

6. The method of any of the preceding claims, herein at least a subset of the sequence of images comprise at least 10,000 molecules per image.

7. The method of any of the preceding claims, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

8. The method of any of the preceding claims, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

9. The method of any of the preceding claims further comprising: labeling molecules within a biological sample; fluorescing the biological sample; and generating the sequence of images while fluorescing the biological sample. 168 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 10. The method of claim 9, wherein the generating of the sequence of images is performed using a microscopy system.

11. The method of any of the preceding claims, wherein the molecules are imaged within living cells.

12. The method of any of the preceding claims further comprising: inferring a probabilistic dynamical model comprising information characterizing the trajectories of the molecules.

13. The method of claim 12, wherein the probabilistic dynamical model comprises a state array and the method further comprises: populating the state array with the information characterizing the trajectories of the molecules.

14. The method of any of the preceding claims further comprising: generating internal metrics of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metrics of confidence.

15. The method of claim 14, wherein the generated internal metrics of confidence is a tracking error rate lower bound that defines a lower bound on a rate of misconnections made by the linking.

16. The method of claim 14, wherein the generated internal metrics comprise: calculating a confidence level for each trajectory. 169 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 17. The method of any of the preceding claims further comprising: generating dynamical metrics independently of specific trajectories.

18. The method of any of the preceding claims, wherein the linking comprises retrieving data comprising a plurality of statistics extracted from a total number of detections or a number of detections in a cell.

19. The method of any of the preceding claims, wherein the providing of data comprises one or more of: visualizing at least a portion of the generated possible trajectories with associated probabilities in a graphical user interface, storing at least a portion of the generated possible trajectories with associated probabilities in physical persistence, loading at least a portion of the generated possible trajectories with associated probabilities in memory, or transmitting at least a portion of the generated possible trajectories with associated probabilities over a network to a remote computing device.

20. The method of any of the preceding claims, wherein at least a portion of the sequence of images comprise contiguous images from a corresponding movie.

21. The method of any of the preceding claims, wherein at least a portion of the sequence of images used by the linking are non-contiguous images from a corresponding movie.

22. A method for single molecule tracking comprising: 170 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) receiving a sequence of images visualizing movement of molecules, the sequence of images comprising a first type generated using a first imaging modality and a second type generated using a second, different imaging modality; detecting spots within the first type of the sequence of images; linking detected spots within the first type of the sequence of images into trajectories using a probabilistic tracking algorithm; segmenting the second type of the sequence of images to generate a plurality of instance masks; assigning molecules within the second type of the sequence of images to at least one instance mask of the plurality of instance masks; and providing data characterizing the linking and assigning to a consuming application or process.

23. The method of claim 22, wherein the probabilistic tracking algorithm comprises a variational Bayesian optimization algorithm.

24. The method of claim 22, wherein the probabilistic tracking algorithm comprises a Gibbs sampling algorithm.

25. The method of claim 22, wherein the partial probabilistic tracking algorithm comprises an adaptive hill climbing algorithm. 171 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 26. The method of any of claims 22 to 25, wherein the first imaging modality and the second imaging modality comprise different molecular labeling techniques.

27. The method of any of claims 22 to 26, wherein the first type of the sequence of images are single molecule tracking (SMT) movies and the second type of the sequence of images are non- SMT movies.

28. The method of any of claims 22 to 27, wherein the detected spots comprise sub-cellular components.

29. The method of any of claims 22 to 28, wherein types of molecules within the first type of the sequence of images are labeled with distinct fluorophores.

30. The method of any of claims 22 to 29, wherein at least a subset of the sequence of images comprise at least 100 molecules per image.

31. The method of any of claims 22 to 30, herein at least a subset of the sequence of images comprise at least 1000 molecules per image.

32. The method of any of claims 22 to 31, herein at least a subset of the sequence of images comprise at least 10,000 molecules per image. 172 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 33. The method of any of claims 22 to 32, wherein the molecules have a density of at least 0.01 emitters per square micron per image.

34. The method of any of claims 22 to 33, wherein the molecules have a density of at least 0.1 emitters per square micron per image.

35. The method of any of claims 22 to 34 further comprising: labeling molecules within a biological sample; fluorescing the biological sample; and generating at least a portion of the sequence of images while fluorescing the biological sample.

36. The method of claim 35, wherein the generating of the sequence of images is performed using a microscopy system.

37. The method of any of claims 22 to 36, wherein the molecules are imaged within living cells.

38. The method of any of claims 22 to 37 further comprising: inferring a probabilistic dynamical model comprising information characterizing the trajectories of the molecules. 173 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 39. The method of claim 38, wherein the probabilistic dynamical model comprises a state array and the method further comprises: populating the state array with the information characterizing the trajectories of the molecules.

40. The method of any of claims 22 to 39 further comprising: generating internal metrics of confidence based on the associated probabilities, wherein the provided data comprises the generated internal metrics of confidence.

41. The method of claim 40, wherein the generated internal metrics of confidence is a tracking error rate lower bound (ERLB) that defines a lower bound on a rate of misconnections made by the linking.

42. The method of claim 40, wherein the generated internal metrics comprise: calculating a confidence level for each trajectory.

43. The method of any of claims 22 to 42 further comprising: generating dynamical metrics independently of specific trajectories.

44. The method of any of claims 22 to 43, wherein the linking comprises retrieving data comprising a plurality of statistics extracted from a total number of detections or a number of detections in a cell. 174 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 45. The method of any of claims 22 to 44, wherein the providing of data comprises one or more of: visualizing at least a portion of the generated possible trajectories with associated probabilities in a graphical user interface, storing at least a portion of the generated possible trajectories with associated probabilities in physical persistence, loading at least a portion of the generated possible trajectories with associated probabilities in memory, or transmitting at least a portion of the generated possible trajectories with associated probabilities over a network to a remote computing device.

46. The method of any of claims 22 to 45, further comprising: generating a plurality of statistical metrics associated with at least one of the trajectories or the at least one instance mask.

47. The method of claim 46 further comprising: storing a hierarchy of instance masks 48. The method of any of claims 22 to 47, wherein at least a portion of the sequence of images comprise contiguous images from a corresponding movie.

49. The method of any of claims 22 to 48, wherein at least a portion of the sequence of images used by the linking are non-contiguous images from a corresponding movie. 175 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 50. The method of any of claims 22 to 49, wherein the detecting utilizes one or more of: a generalized log likelihood ratio spot detector, a difference-of-Gaussians (DoG) detector, a Laplacian-of-Gaussian (LoG) detector, or a determinant of Hessian (DoH) blob detector.

51. The method of any of claims 22 to 50 further comprising: associating the detected spots with spatiotemporal coordinates using subpixel localization.

52. The method of claim 51, wherein the subpixel localization comprises one or more of: a radial symmetry localizer or a maximum likelihood fit to a candidate spot model using the Levenberg-Marquardt method.

53. The method of any of claims 22 to 52, wherein the field of view corresponds to at least a portion of a well.

54. The method of any of the preceding claims, wherein the sequence of images are generated by an apparatus for fluorescence microscopy having: a first optical element or assembly configured to receive a fluorescence excitation light source and produce a collimated light beam having an elongated and linear shape in an x-y plane, wherein the light beam has a uniform intensity across a longer dimension of the linear shape; a second optical element or assembly configured to incline the light beam relative to the z- axis in an x-z plane, wherein the second optical element is further configured to focus the light beam at a sample plane located in the x-y plane, thereby illuminating a portion of the sample plane; 176 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) a third optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam; and a detector device configured to receive light from the illuminated sample plane, wherein the detector device forms one or more projected images based on the light received from the sample plane.

55. The method of claim 54, wherein the first optical component element or assembly comprises a Powell lens to produce the collimated light beam having an elongated and linear shape in an x-y plane.

56. The method of claim 54 or 55, wherein the first optical component element or assembly comprises one or more diffraction gratings to produce the collimated light beam having an elongated and linear shape in an x-y plane.

57. The method of any of claims 54 to 56, wherein the first optical component element or assembly comprises a combination of lenses to produce the collimated light beam having an elongated and linear shape in an x-y plane.

58. The method of any of claims 54 to 57, wherein the second optical component element or assembly comprises an objective. 177 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 59. The method of any of claims 54 to 58, wherein the third optical component element or assembly comprises a galvo mirror configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam.

60. The method of any of claims 54 to 59, wherein the third optical component element or assembly comprises a piezo element configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam.

61. The method of any of claims 54 to 60, wherein the detector device comprises a semiconductor sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

62. The method of any of claims 1 to 53, wherein the sequence of images are generated by a microscopy system for detecting the position of a molecule having: a stage for supporting a sample, wherein the sample contains the molecule; a light source for emitting a light beam capable of inducing a light-based response from the molecule in the sample, wherein the light beam has a linear shape in a sample plane and has a uniform intensity across the longer dimension of the linear shape in the sample plane; an objective lens for focusing the light beam on the sample in the sample plane, wherein the molecule is disposed in the sample plane; and a detector device for monitoring the light-based response from the molecule, thereby detecting the position of the molecule. 178 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 63. The method of claim 62, wherein the microscopy system further comprises a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling a larger total field of view of the microscopy system in the x-y plane.

64. The method of claim 62 or 63, wherein the detector device comprises a semiconductor sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

65. The method of any of claims 62 to 64, wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

66. The method of any of claims 62 to 65, wherein the sample is disposed within an open well of a microplate.

67. The method of claim 66, wherein the microplate comprises a plurality of open wells.

68. The method of any of claim 66 or 67, wherein the microscopy system further comprises: an x-y position controller for altering a field of view of the microscopy system, the altered fields of view encompassing different subsets of the plurality of open wells. 179 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 69. The method of any of claims 66 to 68, wherein the microscopy system further comprises an automated sample-handling robotic system to enable high throughput manipulation of a plurality of samples on the stage, the robotic system comprising: a memory; a processor in communication with the memory; and one or more robotic end-effectors in communication with the processor, wherein the one or more end-effectors manipulate the plurality of samples on the stage based on communication with the processor.

70. A system comprising: at least one data processor; and memory storing instructions, which when executed by at least one data processor, result in operations for implementing a method as in any of claims 1 to 53.

71. The system of claim 70 further comprising: an apparatus for fluorescence microscopy having: a first optical element or assembly configured to receive a fluorescence excitation light source and produce a collimated light beam having an elongated and linear shape in an x-y plane, wherein the light beam has a uniform intensity across a longer dimension of the linear shape; a second optical element or assembly configured to incline the light beam relative to the z-axis in an x-z plane, wherein the second optical element is further configured to focus the 180 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) light beam at a sample plane located in the x-y plane, thereby illuminating a portion of the sample plane; a third optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam; and a detector device configured to receive light from the illuminated sample plane, wherein the detector device forms one or more projected images based on the light received from the sample plane.

72. The system of claim 71, wherein the first optical component element or assembly comprises a Powell lens to produce the collimated light beam having an elongated and linear shape in an x-y plane.

73. The system of claim 71 to 72, wherein the first optical component element or assembly comprises one or more diffraction gratings to produce the collimated light beam having an elongated and linear shape in an x-y plane.

74. The system of any of claims 71 to 73, wherein the first optical component element or assembly comprises a combination of lenses to produce the collimated light beam having an elongated and linear shape in an x-y plane.

75. The system of any of claims 71 to 74, wherein the second optical component element or assembly comprises an objective. 181 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 76. The system of any of claims 71 to 75, wherein the third optical component element or assembly comprises a galvo mirror configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam.

77. The system of any of claims 71 to 76, wherein the third optical component element or assembly comprises a piezo element configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam.

78. The system of any of claims 71 to 77, wherein the detector device comprises a semiconductor sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

79. The system of claim 70 further comprising: a microscopy system for detecting the position of a molecule having: a stage for supporting a sample, wherein the sample contains the molecule; a light source for emitting a light beam capable of inducing a light-based response from the molecule in the sample, wherein the light beam has a linear shape in a sample plane and has a uniform intensity across the longer dimension of the linear shape in the sample plane; an objective lens for focusing the light beam on the sample in the sample plane, wherein the molecule is disposed in the sample plane; and a detector device for monitoring the light-based response from the molecule, thereby detecting the position of the molecule. 182 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 80. The system of claim 79, wherein microscopy system further comprises a scanning optical element or assembly configured to translate the light beam in the sample plane in a direction orthogonal to the longer dimension of the light beam, thereby enabling a larger total field of view of the microscopy system in the x-y plane.

81. The system of claim 79 or 80, wherein the detector device comprises a semiconductor sensor, and wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

82. The system of any of claims 79 to 81, wherein the detector device supports a shutter mode for synchronizing the translation of the light beam in the sample plane with a selective activation or readout of the semiconductor sensor.

83. The system of any of claims 79 to 82, wherein the sample is disposed within an open well of a microplate.

84. The system of claim 83, wherein the microplate comprises a plurality of open wells.

85. The system of any of claims 83 to 84, wherein the microscopy system further comprises: an x-y position controller for altering a field of view of the microscopy system, the altered fields of view encompassing different subsets of the plurality of open wells. 183 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 86. The system of any of claims 73 to 85, wherein the microscopy system further comprises: an automated sample-handling robotic system to enable high throughput manipulation of a plurality of samples on the stage, the robotic system comprising: memory storing instructions; at least one data processor; and one or more robotic end-effectors in communication with the at least one data processor, wherein the one or more end-effectors manipulate the plurality of samples on the stage based on communication with the at least one data processor.

87. A non-transitory computer program product storing instructions, which when executed by at least one data processor forming part of at least one computing device, implement a method as in any of claims 1 to claim 53.

88. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

89. A system comprising: 184 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, using a variational Bayesian optimization algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

90. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, using a Gibbs sampling algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process.

91. A system comprising: means for receiving a sequence of images visualizing movement of molecules; means for linking molecules across the images; means for generating, using an adaptive hill climbing algorithm and based on the linking, possible trajectories for each molecule with associated probabilities; and means for providing data characterizing the generated possible trajectories with associated probabilities to a consuming application or process. 185 NAI-1538920128v2EIK0007 14711-024-228 (101551.228024) 92. A single molecule tracking system comprising: means for receiving a sequence of images visualizing movement of molecules, the sequence of images comprising a first type generated using a first imaging modality and a second type generated using a second, different imaging modality; means for detecting spots within the first type of the sequence of images; means for linking detected spots within the first type of the sequence of images into trajectories using a probabilistic tracking algorithm; means for segmenting the second type of the sequence of images to generate a plurality of instance masks; means for assigning molecules within the second type of the sequence of images to at least one instance mask of the plurality of instance masks; and means for providing data characterizing the linking and assigning to a consuming application or process. 186 NAI-1538920128v2

Images