High frame rate and high dynamic range electron microscopy

Abstract

This application provides high frame rate and high dynamic range electron microscopy. Methods and systems are described for acquiring transmission electron microscope video data on a rolling shutter detector at enhanced frame rates and without temporal distortion. Methods for improving the dynamic range of image and diffraction data acquired using a transmission electron microscope are also described. The disclosed methods and systems may also be applicable to photon detection and imaging systems.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 62 / 872,645, filed July 10, 2019, which application is incorporated herein by reference. (Statement regarding federally sponsored research)

[0002] This invention was made with U.S. Government support under Phase IIA SBIR Award No. DE-SC0013104 from the U.S. Department of Energy. [Background technology]

[0003] A system and method for obtaining video data on a transmission electron microscope (TEM) at high frame rates and / or high dynamic range is described. Like many of the image detectors in other applications, state-of-the-art video detectors in TEMs can be run in either rolling shutter or global shutter timing modes. The "rolling shutter" readout mode offers a duty cycle close to 100% and maximum frame rates, however, because different areas of the camera read out at different times, temporal distortions make it difficult to interpret video of fast moving objects. The "global shutter" readout is synchronized across all pixels and does not suffer from these distortions, however, the duty cycle is reduced as pixels are blocked during readout, thereby resulting in a loss of usable signal while still exposing the sample to the electron beam. The maximum achievable frame rate is also typically slower for global shutter readout. Summary of the Invention [Means for solving the problem]

[0004] Applicants recognize that there exists an unmet need for new methods and systems for acquiring high frame rate and high dynamic range image data that reduce or eliminate temporal distortion while allowing high duty cycle operation. Described in this disclosure are new methods for acquiring video data on a rolling shutter detector at improved frame rates and without temporal distortion. Also described are methods for improving the dynamic range of measurements in electron microscopy. The disclosed methods may also be applicable to photon detection and optical imaging systems.

[0005] Disclosed herein is a method for obtaining transmission electron microscope data comprising: a) providing a transmission electron microscope with a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample positioned at a sample plane, the two-dimensional deflector being positioned between the sample plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected electrons are deflected sequentially onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions; and c) synchronizing the modulation of the deflector control signal with a detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repetitive detector array readout cycles.

[0006] In some embodiments, a method for obtaining transmission electron microscopy data may include: a) providing a transmission electron microscope comprising a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample positioned at a sample plane, the two-dimensional deflector being positioned between the sample plane and the two-dimensional detector array; and b) modulating a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions, wherein the sensitivity of the two or more sub-regions of the two-dimensional detector array, or the dwell time for deflecting electrons onto each of the two or more sub-regions, is individually adjusted.

[0007] Also disclosed is a method for obtaining image data, the method including the steps of: a) providing an optical system including a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object positioned at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or the focal plane and the two-dimensional detector array; b) modulating a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of photons deflected onto each of the two or more sub-regions; and c) synchronizing the modulation of the deflector control signal with a detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repetitive detector array readout cycles.

[0008] In some embodiments, a method for obtaining image data may include the steps of: a) providing an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object positioned at a sample plane or a focal plane, wherein the two-dimensional deflector is positioned between the sample plane or the focal plane and the two-dimensional detector array; and b) modulating a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of photons deflected onto each of the two or more sub-regions, wherein the sensitivity of the two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting photons onto each of the two or more sub-regions are individually adjusted.

[0009] In some embodiments, the two-dimensional deflector is a two-dimensional electrostatic deflector. In some embodiments, the two-dimensional deflector is a two-dimensional electromagnetic deflector. In some embodiments, the two-dimensional deflector is a movable mirror or a prism. In some embodiments, the two or more sub-regions comprise sub-regions of a single two-dimensional detector. In some embodiments, the two or more sub-regions comprise different detectors in a two-dimensional detector array. In some embodiments, the two-dimensional detector array comprises a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, an electronic direct detection (EDD) camera, or a time-of-flight (ToF) image sensor, and the modulation of the deflector control signal is synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that the deflected electrons or photons are directed to the sub-regions that are not being actively read out. In some embodiments, the modulation of the deflector control signal is continuously synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that the deflected electrons or photons are directed to sub-regions that are not being actively read out. In some embodiments, the duty cycle for data acquisition is at least 90% without incurring temporal distortion of the data. In some embodiments, the duty cycle for data acquisition is at least 95% without incurring temporal distortion of the data. In some embodiments, the duty cycle for data acquisition is at least 98% without incurring temporal distortion of the data. In some embodiments, the data sets captured by each of the two or more sub-regions comprise frames of image or diffraction data. In some embodiments, the method may further include processing the image or diffraction data for the two or more sub-regions to generate video data comprising a frame rate that is faster than a readout rate for the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least four times faster than a readout rate for the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video dataset is at least 8 times faster than the readout rate of the CCD, CMOS, or ToF camera.In some embodiments, the frame rate of the video data set is at least 16 times faster than the readout rate of the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least 32 times faster than the readout rate of the CCD, CMOS, or ToF camera. In some embodiments, the switching time for the two-dimensional deflector is less than 10 microseconds. In some embodiments, the switching time for the two-dimensional deflector is less than 1 microsecond. In some embodiments, the switching time for the two-dimensional deflector is less than 100 nanoseconds. In some embodiments, the switching time for the two-dimensional deflector is less than 10 nanoseconds. In some embodiments, the deflector control signal is modulated such that the dwell time for deflecting the electrons or photons into each of the two or more sub-regions is the same. In some embodiments, the deflector control signal is modulated such that the dwell time for deflecting the electrons or photons into each of the two or more sub-regions is different for at least one of the two or more sub-regions. In some embodiments, the dwell time for deflecting the electrons or photons into the two or more sub-regions ranges from about 1 microsecond to about 100 milliseconds. In some embodiments, the deflector control signal is modulated such that the dwell time for deflecting electrons or photons into at least one of the two or more sub-regions is zero, and the data set captured by the at least one sub-region comprises a dark current data set. In some embodiments, each data set captured by the two or more two-dimensional sub-regions comprises image or diffraction data and is further processed to provide a single frame of image or diffraction data having a higher dynamic range than that of the two-dimensional detector array. In some embodiments, the two or more sub-regions of the two-dimensional detector array each comprise an individual detector. In some embodiments, the sensitivity of each of the two or more individual detectors or the saturation level for each of the two or more individual detectors is adjusted individually. In some embodiments, the sensitivity of the two or more individual detectors is adjusted electronically. In some embodiments, the sensitivity of the two or more individual detectors is adjusted by spatial filtering or masking.In some embodiments, the two or more individual detectors comprise two or more detectors with different functionality or different performance specifications. In some embodiments, the two or more detectors comprise a CCD camera, a CMOS camera, a dark field STEM detector, a Faraday cup, a quad photodiode, an annular dark field detector, a bright field detector, a general purpose detector, or any combination thereof. In some embodiments, at least one of the two or more individual detectors is positioned at a different distance from the sample plane or the focal plane. In some embodiments, the data sets captured by the two or more detectors are compiled to generate a multi-modal data set comprising image data, electron beam current data, electron beam alignment data, segmented dark field data, or any combination thereof. In some embodiments, the data sets captured by the two or more sub-regions are processed to correct for distortions introduced by the deflection step.

[0010] Disclosed herein is a system for obtaining transmission electron microscopy data comprising: a) a transmission electron microscope; b) a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample, the two-dimensional deflector being positioned within the transmission electron microscope between the sample plane and a two-dimensional detector array; and c) a deflector control system configured to modulate a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions, the deflector control signal being synchronized with a detector array readout signal such that each of the two or more two-dimensional sub-regions captures a data set within each of a series of repeating detector array readout cycles.

[0011] In some embodiments, a disclosed system for obtaining transmission electron microscopy data may include: a) a transmission electron microscope; b) a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample, the two-dimensional deflector being positioned within the transmission electron microscope between the sample plane and the two-dimensional detector array; and c) a deflector control system configured to modulate a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions, wherein the sensitivity of the two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting electrons onto each of the two or more sub-regions is individually adjustable.

[0012] Also disclosed herein is a system for obtaining image data, comprising: a) an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object positioned at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or focal plane and the two-dimensional detector array; and b) a deflector control system configured to modulate a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of photons deflected onto each of the two or more sub-regions, the deflector control signal being synchronized with the detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repetitive detector array readout cycles.

[0013] In some embodiments, the disclosed system for acquiring image data may comprise: a) an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object positioned at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or the focal plane and the two-dimensional detector array; and b) a deflector control system configured to modulate a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of photons deflected onto each of the two or more sub-regions, wherein the sensitivity of the two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting photons onto each of the two or more sub-regions is individually adjustable.

[0014] In some embodiments, the system further comprises a processor configured to collect and analyze the data sets captured by the two or more sub-regions. In some embodiments, the two-dimensional deflector is a two-dimensional electrostatic deflector. In some embodiments, the two-dimensional deflector is a two-dimensional electromagnetic deflector. In some embodiments, the two-dimensional deflector is a movable mirror or a prism. In some embodiments, the two or more sub-regions comprise sub-regions of a single two-dimensional detector. In some embodiments, the two or more sub-regions comprise different detectors in a two-dimensional detector array. In some embodiments, the two-dimensional detector array comprises a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, an electron direct detection (EDD) image sensor, or a time-of-flight (ToF) image sensor, and the modulation of the deflector control signal is synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that the deflected electrons or photons are directed to the sub-regions that are not being actively read out. In some embodiments, the modulation of the deflector control signal is continuously synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that the deflected electrons or photons are directed to a small area that is not being actively read out. In some embodiments, the duty cycle for data acquisition is at least 90% without incurring temporal distortion of the data. In some embodiments, the duty cycle for data acquisition is at least 90% without incurring temporal distortion of the data. The duty cycle for data acquisition is at least 95% without incurring distortion. In some embodiments, the duty cycle for data acquisition is at least 98% without incurring temporal distortion of the data. In some embodiments, each data set captured by the two or more sub-regions comprises a single frame of image or diffraction data. In some embodiments, the image or diffraction data for the two or more sub-regions are processed to generate a video data set comprising a frame rate faster than the readout rate for the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least 4 times faster than the frame rate of the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least 8 times faster than the frame rate of the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least 16 times faster than the frame rate of the CCD, CMOS, or ToF camera. In some embodiments, the frame rate of the video data set is at least 32 times faster than the frame rate of the CCD, CMOS, or ToF camera. In some embodiments, the switching time for the two-dimensional deflector is less than 10 microseconds. In some embodiments, the switching time for the two-dimensional deflector is less than 1 microsecond. In some embodiments, the switching time for the two-dimensional deflector is less than 100 nanoseconds. In some embodiments, the switching time for the two-dimensional deflector is less than 10 nanoseconds. In some embodiments, the deflector control signal is modulated such that the dwell time for the deflected electrons or photons in each of the two or more sub-regions is the same. In some embodiments, the deflector control signal is modulated such that the dwell time for the deflected electrons or photons in each of the two or more sub-regions is different with respect to at least one of the two or more sub-regions. In some embodiments, the dwell time for deflecting the electrons or photons into the two or more sub-regions ranges from about 1 microsecond to about 100 milliseconds.In some embodiments, the deflector control signal is modulated such that the dwell time for deflecting electrons or photons into at least one of the two or more sub-regions is zero, and the data set captured by the at least one sub-region comprises a dark current data set. In some embodiments, each data set captured by the two or more sub-regions comprises image or diffraction data and is further processed to provide a single frame of image or diffraction data having a higher dynamic range than that of the two-dimensional detector array. In some embodiments, the two or more sub-regions of the two-dimensional detector array each comprise an individual detector. In some embodiments, the sensitivity of each of the two or more individual detectors, or the saturation level for each of the two or more individual detectors, is individually adjustable. In some embodiments, the sensitivity of the two or more individual detectors is adjusted electronically. In some embodiments, the sensitivity of the two or more individual detectors is adjusted by spatial filtering or masking. In some embodiments, the two or more individual detectors comprise two or more detectors with different functionality or different performance specifications. In some embodiments, the two or more different detectors comprise a CCD camera, a CMOS camera, a dark field STEM detector, a Faraday cup, a quad photodiode, an annular dark field detector, a bright field detector, a general purpose detector, or any combination thereof. In some embodiments, at least one of the two or more individual detectors is positioned at a different distance from the sample plane or the focal plane. In some embodiments, the data sets captured by the two or more detectors are compiled to generate a multi-modal data set comprising image data, electron beam current data, electron beam alignment data, segmented dark field data, or any combination thereof. In some embodiments, the data sets captured by the two or more sub-regions are processed to correct for distortions introduced by the deflection step. The present invention provides, for example, the following: (Item 1) 1. A method for obtaining transmission electron microscopy data, the method comprising: a) providing a transmission electron microscope comprising a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample positioned at a sample plane, the two-dimensional deflector being positioned between the sample plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the electrons deflected onto each of the two or more sub-regions; c) synchronizing modulation of the deflector control signals with a detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repeating detector array readout cycles; A method comprising: (Item 2) 1. A method for obtaining transmission electron microscopy data, the method comprising: a) providing a transmission electron microscope comprising a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample positioned at a sample plane, the two-dimensional deflector being positioned between the sample plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the electrons deflected onto each of the two or more sub-regions; Including, A method wherein the sensitivity of two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting electrons into each of the two or more sub-regions is individually adjusted. (Item 3) 1. A method for obtaining image data, the method comprising: a) providing an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected or scattered by an object located at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or focal plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the photons deflected onto each of the two or more sub-regions; c) synchronizing modulation of the deflector control signals with a detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repeating detector array readout cycles; A method comprising: (Item 4) 1. A method for obtaining image data, the method comprising: a) providing an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected or scattered by an object located at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or focal plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the photons deflected onto each of the two or more sub-regions; Including, A method wherein the sensitivity of two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting photons to each of the two or more sub-regions is individually adjusted. (Item 5) 3. The method according to claim 1, wherein the two-dimensional deflector is a two-dimensional electrostatic deflector. (Item 6) 3. The method according to claim 1, wherein the two-dimensional deflector is a two-dimensional electromagnetic deflector. (Item 7) 5. The method according to claim 3, wherein the two-dimensional deflector is a movable mirror or a prism. (Item 8) 8. The method of any one of claims 1-7, wherein the two or more sub-regions comprise sub-regions of a single two-dimensional detector. (Item 9) 8. The method of any one of claims 1-7, wherein the two or more sub-regions comprise different detectors in a two-dimensional detector array. (Item 10) 9. The method of any one of items 1, 3, 5, 6, 7, or 8, wherein the two-dimensional detector array comprises a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, an electron direct detection (EDD) camera, or a time-of-flight (ToF) image sensor, and the modulation of the deflector control signal is synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that deflected electrons or photons are directed to a small area that is not actively being read out. (Item 11) 11. The method of claim 10, wherein the duty cycle for data acquisition is at least 90% without incurring time distortion of said data. (Item 12) 11. The method of claim 10, wherein the duty cycle for data acquisition is at least 95% without incurring time distortion of said data. (Item 13) 11. The method of claim 10, wherein the duty cycle for data acquisition is at least 98% without incurring time distortion of said data. (Item 14) 14. The method of any one of claims 10-13, wherein the data set captured by each of the two or more sub-regions comprises a frame of image or diffraction data. (Item 15) 15. The method of claim 14, further comprising processing the image or diffraction data for the two or more small regions to generate video data with a frame rate faster than a readout rate for the CCD, CMOS, or ToF camera. (Item 16) 16. The method of claim 15, wherein the frame rate of the video dataset is at least four times faster than the readout rate of the CCD, CMOS, or ToF camera. (Item 17) 16. The method of claim 15, wherein the frame rate of the video dataset is at least 8 times faster than the readout rate of the CCD, CMOS, or ToF camera. (Item 18) Item 16. The method of item 15, wherein the frame rate of the video dataset is at least 16 times faster than the readout rate of the CCD, CMOS, or ToF camera. (Item 19) 16. The method of claim 15, wherein the frame rate of the video dataset is at least 32 times faster than the readout rate of the CCD, CMOS, or ToF camera. (Item 20) 20. The method of any one of claims 1-19, wherein the switching time for the two-dimensional deflector is less than 10 microseconds. (Item 21) 20. The method of any one of claims 1-19, wherein the switching time for the two-dimensional deflector is less than 1 microsecond. (Item 22) 20. The method of any one of claims 1-19, wherein the switching time for the two-dimensional deflector is less than 100 nanoseconds. (Item 23) 20. The method of any one of claims 1-19, wherein the switching time for the two-dimensional deflector is less than 10 nanoseconds. (Item 24) 24. The method of any one of claims 1-23, wherein the deflector control signal is modulated such that the dwell time for deflecting electrons or photons into each of the two or more sub-regions is identical. (Item 25) 25. The method of any one of claims 1-24, wherein the deflector control signal is modulated such that the dwell time for deflecting electrons or photons into each of the two or more sub-regions is different for at least one of the two or more sub-regions. (Item 26) 26. The method according to any one of items 1-25, wherein the dwell time for deflecting electrons or photons into the two or more small regions ranges from about 1 microsecond to about 100 milliseconds. (Item 27) 27. The method of any one of claims 1-26, wherein the deflector control signal is modulated such that a dwell time for deflecting electrons or photons into at least one of the two or more sub-regions is zero, and the data set captured by the at least one sub-region comprises a dark current data set. (Item 28) 28. The method of any one of items 2, 4, 25, 26, or 27, wherein each data set captured by the two or more two-dimensional sub-regions comprises image or diffraction data and is further processed to provide a single frame of image or diffraction data having a higher dynamic range than that of the two-dimensional detector array. (Item 29) 29. The method of any one of claims 1-28, wherein each of the two or more sub-regions of the two-dimensional detector array comprises an individual detector. (Item 30) 30. The method of claim 29, wherein the sensitivity of each of the two or more individual detectors or the saturation level for each of the two or more individual detectors is adjusted individually. (Item 31) 31. The method of claim 30, wherein the sensitivity of the two or more individual detectors is electronically adjusted. (Item 32) 31. The method of claim 30, wherein the sensitivity of the two or more individual detectors is adjusted by spatial filtering or masking. (Item 33) 33. The method of any one of items 29-32, wherein the two or more individual detectors comprise two or more detectors with different functionality or different performance specifications. (Item 34) Item 34. The method of item 33, wherein the two or more detectors comprise a CCD camera, a CMOS camera, a dark field STEM detector, a Faraday cup, a quad photodiode, an annular dark field detector, a bright field detector, a general purpose detector, or any combination thereof. (Item 35) 35. The method of any one of items 28-34, wherein at least one of the two or more individual detectors is positioned at a different distance from the sample plane or focal plane. (Item 36) 36. The method of any one of claims 33-35, wherein the data sets captured by the two or more detectors are compiled to generate a multi-modal data set comprising image data, electron beam current data, electron beam alignment data, segmented dark field data, or any combination thereof. (Item 37) Item 37. The method according to any one of items 1-36, wherein the data sets captured by the two or more sub-regions are processed to correct for distortion introduced by the deflection step. (Item 38) 1. A system for obtaining transmission electron microscopy data, the system comprising: a) a transmission electron microscope; b) a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample, said two-dimensional deflector being positioned within said transmission electron microscope between a sample plane and a two-dimensional detector array; c) a deflector control system configured to modulate deflector control signals such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions; Equipped with The system, wherein the deflector control signals are synchronized with a detector array readout signal such that each of the two or more two-dimensional sub-regions captures a data set within each of a series of repeating detector array readout cycles. (Item 39) 1. A system for obtaining transmission electron microscopy data, the system comprising: a) a transmission electron microscope; b) a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample, said two-dimensional deflector being positioned within said transmission electron microscope between a sample plane and a two-dimensional detector array; c) a deflector control system configured to modulate deflector control signals such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions; Equipped with A system in which the sensitivity of two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting electrons into each of the two or more sub-regions is individually adjustable. (Item 40) 1. A system for obtaining image data, the system comprising: a) an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object located at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or focal plane and a two-dimensional detector array; b) a deflector control system configured to modulate a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the photons deflected onto each of the two or more sub-regions; Equipped with The deflector control signals are synchronized with a detector array readout signal such that each of the two or more sub-regions captures a data set within each of a series of repeating detector array readout cycles. (Item 41) 1. A system for obtaining image data, the system comprising: a) an optical system comprising a two-dimensional deflector configured to deflect photons transmitted, reflected, or scattered by an object located at a sample plane or a focal plane, the two-dimensional deflector being positioned between the sample plane or focal plane and a two-dimensional detector array; b) a deflector control system configured to modulate a deflector control signal such that the deflected photons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the photons deflected onto each of the two or more sub-regions; Equipped with A system in which the sensitivity of two or more sub-regions of the two-dimensional detector array or the dwell time for deflecting photons to each of the two or more sub-regions is individually adjustable. (Item 42) 42. The system of any one of claims 39-41, further comprising a processor configured to collect and analyze the data set captured by the two or more small regions. (Item 43) 40. The system of any one of claims 38 or 39, wherein the two-dimensional deflector is a two-dimensional electrostatic deflector. (Item 44) 40. The system of any one of claims 38 or 39, wherein the two-dimensional deflector is a two-dimensional electromagnetic deflector. (Item 45) 42. The system of any one of claims 40 or 41, wherein the two-dimensional deflector is a movable mirror or a prism. (Item 46) 46. ​​The system of any one of claims 39-45, wherein the two or more sub-regions comprise sub-regions of a single two-dimensional detector. (Item 47) 46. ​​The system of any one of claims 39-45, wherein the two or more sub-regions comprise different detectors in a two-dimensional detector array. (Item 48) 47. The system of any one of items 38, 40, 42, 43, 44, 45, or 46, wherein the two-dimensional detector array comprises a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, an electron direct detection (EDD) image sensor, or a time-of-flight (ToF) image sensor, and the modulation of the deflector control signal is synchronized with a rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that deflected electrons or photons are directed to a small area that is not actively being read out. (Item 49) Item 49. The system of item 48, wherein the duty cycle for data acquisition is at least 90% without incurring time distortion of the data. (Item 50) Item 49. The system of item 48, wherein the duty cycle for data acquisition is at least 95% without incurring time distortion of the data. (Item 51) Item 49. The system of item 48, wherein the duty cycle for data acquisition is at least 98% without incurring time distortion of the data. (Item 52) 52. The system of any one of claims 48-51, wherein each data set captured by the two or more sub-regions comprises a single frame of image or diffraction data. (Item 53) 53. The system of claim 52, wherein the image or diffraction data for the two or more small regions is processed to generate a video data set having a frame rate faster than a readout rate for the CCD, CMOS, or ToF camera. (Item 54) Item 54. The system of item 53, wherein the frame rate of the video dataset is at least four times faster than the frame rate of the CCD, CMOS, or ToF camera. (Item 55) Item 54. The system of item 53, wherein the frame rate of the video dataset is at least 8 times faster than the frame rate of the CCD, CMOS, or ToF camera. (Item 56) Item 54. The system of item 53, wherein the frame rate of the video dataset is at least 16 times faster than the frame rate of the CCD, CMOS, or ToF camera. (Item 57) Item 54. The system of item 53, wherein the frame rate of the video dataset is at least 32 times faster than the frame rate of the CCD, CMOS, or ToF camera. (Item 58) 58. The system of any one of claims 38-57, wherein the switching time for the two-dimensional deflector is less than 10 microseconds. (Item 59) 58. The system of any one of claims 38-57, wherein the switching time for the two-dimensional deflector is less than 1 microsecond. (Item 60) 58. The system of any one of claims 38-57, wherein the switching time for the two-dimensional deflector is less than 100 nanoseconds. (Item 61) 58. The system of any one of claims 38-57, wherein the switching time for the two-dimensional deflector is less than 10 nanoseconds. (Item 62) 62. The system of any one of claims 38-61, wherein the deflector control signal is modulated such that the residence time for the deflected electrons or photons in each of the two or more sub-regions is identical. (Item 63) 62. The system of any one of claims 38-61, wherein the deflector control signal is modulated such that a residence time for the deflected electrons or photons in each of the two or more sub-regions is different with respect to at least one of the two or more sub-regions. (Item 64) 64. The system of any one of items 38-63, wherein a dwell time for deflecting electrons or photons into the two or more small regions ranges from about 1 microsecond to about 100 milliseconds. (Item 65) 65. The system of any one of claims 38-64, wherein the deflector control signal is modulated such that a dwell time for deflecting electrons or photons into at least one of the two or more sub-regions is zero, and the data set captured by the at least one sub-region comprises a dark current data set. (Item 66) 66. The system of any one of items 39, 41, 63, 64, or 65, wherein each data set captured by the two or more sub-regions comprises image or diffraction data and is further processed to provide a single frame of image or diffraction data having a higher dynamic range than that of the two-dimensional detector array. (Item 67) 67. The system of any one of claims 38-66, wherein each of the two or more sub-regions of the two-dimensional detector array comprises an individual detector. (Item 68) Item 68. The system of item 67, wherein the sensitivity of each of the two or more individual detectors or the saturation level for each of the two or more individual detectors is individually adjustable. (Item 69) Item 70. The system of item 68, wherein the sensitivity of the two or more individual detectors is electronically adjusted. (Item 70) Item 69. The system of item 68, wherein the sensitivity of the two or more individual detectors is adjusted by spatial filtering or masking. (Item 71) 71. The system of any one of items 67-70, wherein the two or more individual detectors comprise two or more detectors with different functionality or different performance specifications. (Item 72) Item 72. The system of item 71, wherein the two or more different detectors comprise a CCD camera, a CMOS camera, a dark field STEM detector, a Faraday cup, a quad photodiode, an annular dark field detector, a bright field detector, a general purpose detector, or any combination thereof. (Item 73) 73. The system of any one of claims 67-72, wherein at least one of the two or more individual detectors is positioned at a different distance from the sample plane or focal plane. (Item 74) 73. The system of any one of items 71-72, wherein the datasets captured by the two or more detectors are compiled to generate a multi-modal dataset comprising image data, electron beam current data, electron beam alignment data, segmented dark field data, or any combination thereof. (Item 75) 74. The system of any one of claims 38-73, wherein the data sets captured by the two or more sub-regions are processed to correct for distortion introduced by the deflection step. (Incorporated by reference)

[0015] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference in their entirety to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference in its entirety. In the event of a conflict between a term in this specification and a term in an incorporated reference, the term in this specification shall control. [Brief description of the drawings]

[0016] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings.

[0017] [Figure 1] FIG. 1 provides a non-limiting example of a transmission electron microscope (TEM) configured to obtain high frame rate video data according to one aspect of the present disclosure.

[0018] [Diagram 2] FIG. 2 provides an illustration of one non-limiting example of an exposure sequence for deflecting electrons (or, in some cases, photons) to different sub-regions of a two-dimensional detector.

[0019] [Diagram 3] FIG. 3 provides a schematic diagram of a two-dimensional electrostatic deflector.

[0020] [Figure 4] Figures 4A-E provide non-limiting examples of timing diagrams for synchronizing electrostatic deflection of electrons in a TEM to expose different sub-regions of a two-dimensional detector, such as a CMOS camera, with a rolling shutter readout signal for the two-dimensional detector. Figure 4A is a schematic diagram of a readout area for a CMOS image sensor. Figure 4B is a timing diagram of row-by-row readout from the image sensor. Figure 4C is a sub-region of the image sensor as defined by a two-dimensional deflector. Figure 4D is a timing diagram of the sequence of exposures illustrated in Figure 4C. Figure 4E is a timing diagram of voltages applied between electrode pairs in a two-dimensional electrostatic deflector to implement the deflection patterns and timing depicted in Figures 4C and 4D.

[0021] [Diagram 5] FIG. 5 provides an example of a typical selected area electron diffraction (SAED) electron diffraction pattern showing the saturation effect in the central spot.

[0022] [Figure 6] FIG. 6 provides a schematic diagram of a high dynamic range (HDR) transmission electron microscope.

[0023] [Figure 7] FIG. 7 provides a non-limiting example of a flow chart for obtaining HDR data.

[0024] [Figure 8] FIG. 8 provides a non-limiting example of a flow chart for a data analysis method used to process a multi-detector data set.

[0025] [Figure 9] FIG. 9 provides a schematic diagram of a scanning transmission electron microscope (STEM) scan controller combined with a two-dimensional electrostatic deflector to deflect electrons transmitted or scattered by the sample to different detectors or sub-regions of the detector.

[0026] [Figure 10] 10A-E provide non-limiting examples of timing diagrams for controlling electrostatic deflection of electrons in a TEM and exposing different sub-regions of a two-dimensional detector or different detectors in a two-dimensional array of detectors. FIG. 10A is a schematic diagram of a probe beam location on a sample. FIG. 10B is a timing diagram of the scanning of a probe beam across a sample. FIG. 10C is a schematic diagram of an array of ten individual detectors. In some cases, the detection regions 1-9 may be sub-regions of a single detector, e.g., a CMOS camera. FIG. 10D is a timing diagram of the sequence of exposures illustrated in FIG. 10C, with different exposure times (dwell times) for different detectors / detector regions. FIG. 10E is a timing diagram of the voltages applied between electrode pairs in a two-dimensional electrostatic deflector to implement the deflection patterns and timing depicted in FIGS. 10C and 10D.

[0027] [Figure 11]FIG. 11 provides a schematic diagram of a computer system that is programmed or otherwise configured to implement the methods and systems provided herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Detailed Description Methods and systems for the acquisition of high frame rate (HFR) video data and / or high dynamic range (HDR) image or diffraction data in transmission electron microscopy are described. In some cases, the disclosed methods and systems may be applied to other image data acquisition modes, such as optical microscopy or optical imaging. The disclosed methods and systems share a common feature in that in each case, a high-speed two-dimensional deflector is used to sequentially deflect electrons or photons transmitted or scattered by the sample (or object) to each of a series of sub-regions of a two-dimensional detector with precise control of timing and spatial positioning. In some cases, the sub-regions may comprise subdivisions of a single two-dimensional detector, e.g., quadrants of a single CCD or CMOS image sensor. In some cases, the sub-regions may comprise individual detectors, e.g., individual CCD or CMOS image sensors in a two-dimensional array of detectors.

[0029] For high frame rate video data acquisition, rapid switching of a two-dimensional deflector between different sub-regions of a detector, e.g., a CMOS camera sensor, and precise timing and continuous synchronization of the two-dimensional deflector with the rolling shutter readout of the CMOS camera provide continuous acquisition of video data at a frame rate higher than the readout rate for the CMOS camera. The disclosed methods and systems may include, for example, the use of a high-speed two-dimensional electrostatic or electromagnetic deflector to direct electrons to sub-regions of the camera that are not actively being readout, such that the camera's image sensor is divided into multiple sub-frames that acquire images or diffraction patterns at different times. These sub-frames may be continuously synchronized to the camera's rolling shutter such that each sub-region is exposed between readouts of the camera's image sensor, and subsequently assembled to provide continuous video data at a frame rate higher than the camera image sensor readout rate. The resulting frame rate and temporal resolution of the compiled video data is equal to the camera readout rate multiplied by the number of sub-frames collected in a full frame image. For example, an image sensor or detector that is subdivided into an 8x8 array of sub-frames with a high speed two-dimensional deflector produces video data with a 64-fold improvement in frame rate and temporal resolution.

[0030] For high dynamic range data acquisition, fast switching and adjustable dwell times for deflecting electrons or photons to each of the detector sub-regions in, for example, a CMOS camera provide for acquisition of a series of image data sets within each cycle of CMOS image sensor readout that can be subsequently processed to generate one or more image frames with a much higher dynamic range than would be possible with a single full frame data capture. In some cases, for example, a single detector divided into 16 exposure regions can be used to acquire images under 16 different exposure conditions such that the detector readout is processed to produce a single HDR image. In some cases, a single detector divided into 16 exposure regions can be used to acquire four images under each of four different exposure conditions such that the detector readout is processed to produce four HDR images. In some cases, the data acquired during each detector array readout cycle can be processed to produce 1, 2, 4, 8, 16, 32, 64, 128, or more HDR images.

[0031] In some cases, high dynamic range data acquisition may be provided by using an array of individual detectors, e.g., an array of CMOS image sensors, where the sensitivity and / or saturation level for each individual detector may be adjusted on a detector-by-detector basis instead of or in addition to adjusting the dwell time for the deflection of electrons or photons to each individual detector. In some cases, the disclosed methods and systems may include the use of high-speed electrostatic deflectors that may sequentially direct scattered and transmitted electrons to different sub-regions of the detector, or to each of a series of detectors. In some cases, a deflector control system exposes each detector for different lengths of time, and a processor or computer collects and analyzes data from the sub-regions or series of detectors. In some cases, the deflector control system is integrated with timing for reading out the detectors and / or modulating operating parameters of the electron microscope, such as the location or angle of the probe beam on the sample. The analysis method implemented by the processor or computer forms a composite image from data acquired during different sub-regions or detector readouts representing different exposure times and / or sensitivity levels. The resulting composite image may thus have a substantially wider dynamic range than individual image readouts from the individual sub-regions or detectors.

[0032] In some cases, the disclosed systems may be configured for simultaneous acquisition of high frame rate and high dynamic range video data with image data or diffraction data. Although primarily discussed in the context of electron microscopy, the disclosed methods and systems may also be applied to optical imaging, where a suitable high speed two-dimensional deflector for photons, e.g., a rotating mirror or prism, is substituted for the two-dimensional deflector used to deflect electrons. Various aspects of the disclosed methods and systems described herein may be applied to any of the specific electron or photon microscopy applications described below, or for other types of image, diffraction pattern, or spectral data acquisition applications. It should be understood that different aspects of the disclosed methods and systems may be understood individually, collectively, or in combination with one another.

[0033] Definitions: Unless defined otherwise, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0034] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Any reference to "or" herein is intended to include "and / or" unless stated otherwise.

[0035] As used herein, the term "about" a number refers to that number plus or minus 10% of that number. When used in the context of a range, the term "about" refers to that range from minus 10% of its minimum value to plus 10% of its maximum value.

[0036] As used herein, the term "two-dimensional detector array" may, in some cases, refer to a single detector, e.g., a CCD or CMOS image sensor comprising a two-dimensional array of individual pixels or groups of pixels, or in some cases may refer to a two-dimensional array of individual detectors, e.g., a two-dimensional array of CCD or CMOS image sensors. In some cases, the two-dimensional detector array may comprise a two-dimensional array of individual detectors, including a CMOS image sensor, a CCD image sensor, a dark-field STEM detector, a Faraday cup sensor, or a combination of other types of detectors, as will be discussed in more detail below.

[0037] As used herein, the term "video" may, in some cases, refer to image or diffraction pattern data organized as a function of time. In some cases, the term "video" may refer to image or diffraction data organized as a function of different measurement operating parameters, such as electron probe beam position, or as a function of a multi-dimensional vector of measurement operating parameters to map the data onto a high-dimensional data structure. In any of these cases, the data may be referred to as "high frame rate data."

[0038] As used herein, the phrase "continuous synchronization" refers to precise synchronization of two processes (e.g., deflection of a two-dimensional deflector with a rolling shutter readout from a CMOS camera) over a long-term time period (e.g., a data acquisition period of up to 8 hours or more) such that the timing accuracy for the two processes meets defined performance specifications (e.g., the synchronization of the two processes is accurate to within 50 milliseconds, or better), as will be discussed in more detail below.

[0039] High Frame Rate Electron Microscopy: In transmission electron microscopy (TEM), high-speed video is an increasingly important tool for characterizing the dynamics of biological samples and nanoscale materials. When synchronized with modulation of experimental parameters such as sample tilt or probe beam position, high-speed video enables high-throughput acquisition of large multidimensional data sets that are useful in scientific and industrial applications.

[0040] The electrons used in the TEM imaging process can easily damage the sample being measured. An ideal detection process would utilize a 100% duty cycle for image capture, avoiding unnecessary or unmeasured exposure of the sample. This makes a global shutter operation undesirable, as electrons would continue to strike the sample during all or part of the readout cycle and be discarded rather than detected. By directing electrons transmitted or scattered by the sample (e.g., the pattern of transmitted or scattered electrons that comprise the image or diffraction data) away from the area of ​​the camera being actively readout in a rolling shutter mode and towards those areas that are available for exposure, the methods and systems disclosed herein avoid wasting electrons by ensuring that every electron that strikes the sample has an opportunity to contribute to the measurement.

[0041] In addition to damaging the sample being measured, the electron beam in a TEM can easily damage conventional image detectors. Therefore, image detectors used in TEMs must be radiation hardened or specifically designed for use with electron beams, leading to significant trade-offs in durability, spatial resolution, dynamic range, and speed. As noted above, there is a need for improved methods of capturing high frame rate TEM video data. The methods and systems disclosed herein can be used to improve the performance of most TEM image sensors currently on the market, as well as detectors currently under development.

[0042] There are some applications in TEM imaging where a reduced number of pixels per video frame is an acceptable trade-off in exchange for improved frame rate. Most TEM image sensors currently on the market offer the ability to read out a reduced field of view or bin pixels together, resulting in a loss of spatial resolution in exchange for faster speed. The disclosed method and system provide additional flexibility for users to make similar trade-offs across a wider range of parameters and obtain substantially higher frame rates.

[0043] Prior art disclosures in both photon and electron optics have utilized two-dimensional electrostatic deflectors and subdivided image sensors for the purpose of recording burst or streak images. See, for example, U.S. Pat. Nos. 3,887,841 and 4,565,923 (which describe a CCD framing camera technique) and U.S. Pat. No. 9,451,177 B2 (which describes a high-speed camera in which light is deflected onto a series of phosphor blocks and converted into electrons that are converted back to light for the capture of a sequence of image frames). In the case of electron optics, two-dimensional electrostatic deflectors have been used to capture multiple subframes of a strobed-illuminated image within a single camera readout, as described in U.S. Pat. No. 9,165,743 B2. To our knowledge, none of these references describe the continuous synchronization of high-speed two-dimensional electrostatic or electromagnetic deflectors with a rolling shutter video image sensor to achieve continuous high frame rate video data acquisition of arbitrary length, which is the primary advantage of the disclosed method.

[0044] A somewhat related technique for performing spectroscopic measurements in an electron microscope is described in International Publication No. WO2019 / 028129A1. In this publication, a CMOS sensor designed to capture electron images is used to record electron energy loss spectra (EELS). The spectrum is projected onto a camera in long, thin stripes. A one-dimensional electrostatic deflector scrolls the spectral exposure stripes across the camera as a function of time, so as to avoid exposing any area of ​​the image sensor that is actively being read out. This is intended to enable EELS spectral data acquisition at a higher throughput. The method described in the International Publication No. WO2019 / 028129A1 reference is distinct from that disclosed herein in at least three ways. That is, (i) the method disclosed herein allows for improved acquisition of video data at high frame rates for real space images and diffraction data, whereas the method of WO2019 / 028129A1 is applied to EELS spectra, (ii) the method disclosed herein uses a two-dimensional deflector and arranges video frames in a grid-like pattern on a two-dimensional sensor, whereas the method of WO2019 / 028129A1 only shifts the spectral exposure stripes in one dimension, and (iii) the method disclosed herein applies to a broader definition of "sensor". While the method disclosed in WO2019 / 028129A1 specifies the use of a CMOS electronic image detector, the method disclosed herein may be applied to deflect the pattern of transmitted or scattered electrons to different regions of a single image detector or between elements in an array of different detectors, where different detectors may have different sensitivity levels or detection capabilities, for example, in the case of high dynamic range data acquisition as will be discussed in more detail below.

[0045] As noted above, the disclosed methods and systems improve video frame rate data acquisition in electron microscopes (or in optical systems as will be discussed in more detail below) by sequentially sequencing the exposure of different detectors or different sub-areas of a single detector such that there is no loss of signal during readout. This is accomplished using a high-speed two-dimensional deflector, e.g., a 2D electrostatic or 2D electromagnetic deflector, whose switching time is much faster than the dwell time used to expose each sub-area of ​​the detector and direct the pattern of electrons (e.g., image data or diffraction data) transmitted or scattered by the sample to the different sub-areas of the detector, as illustrated in FIG. 1. FIG. 1 shows a non-limiting example of a TEM in which an electron beam originating from an electron gun or other source located at the top of the instrument propagates downward through the sample. Electron optics (not shown) images the beam transmitted or scattered by the sample onto a detector located at the bottom of the instrument. Between the sample and the detector (as configured in a standard TEM) is a 2D deflector device that applies an electric field and shifts or deflects the transmitted or scattered electrons (e.g., image or diffraction pattern) to a programmed sub-area of ​​the detector. Electronic hardware and automation software are used to control the programmed sequence of sub-area exposures and to maintain precise continuous synchronization between the deflection sequence and the camera readout rate (e.g., by modulating the deflector control signal). Although FIG. 1 depicts both the “STEM Scan Control” and “Detector Synchronization” modules as being directly connected to the “Subframe Sequence Control” module, the interconnection of these timing systems may be configured differently in other implementations. The “Detector Synchronization” module may, for example, be directly connected to the “STEM Scan Control” module. In the example depicted in FIG. 1, the camera is subdivided into 16 exposure sub-frames. If the detector is a single standard TEM camera, continuous video with a 16-fold increase in frame rate is achieved.

[0046] Figure 2 provides a non-limiting example of an exposure sequence that divides the detector into nine distinct sub-regions. The square outline represents the total detector area at a given time. The dashed lines represent pixels being read out at the same time. In rolling shutter readout mode, the readout line moves continuously across the camera and immediately returns to the first line after the last line is read out. The small black squares indicate the exposed regions at the same time. The deflector and camera timing are carefully synchronized so that the exposed regions never coincide with the readout regions.

[0047] FIG. 3 provides a schematic diagram of a two-dimensional electrostatic deflector with two pairs of electrodes (X and Y electrodes) that can be used in implementing the disclosed methods and systems. FIGS. 4A-E provide non-limiting examples of timing diagrams for synchronizing electrostatic deflection of electrons in a TEM to expose different sub-areas of a two-dimensional detector, such as a CMOS camera, with a rolling shutter readout signal for the two-dimensional detector. In this illustration, the readout of the CMOS camera sensor is depicted with respect to rows of pixels in three areas A, B, and C (FIG. 4A) in the upper right timing diagram (FIG. 4B). The readout of the image sensor proceeds row by row across the detector in one dimension and then returns to the first row. In this example, the two-dimensional deflector deflects electrons into a grid of 3×3=9 subframes or exposure areas (FIG. 4C) according to the sequence of exposure depicted in the timing diagram of FIG. 4D, thereby resulting in a nine-fold improvement in the video frame rate. A sequence of voltages applied between two pairs of electrodes (i.e., X and Y electrodes in the two-dimensional electrostatic deflector illustrated in FIG. 3) used to implement the deflection patterns and timing shown in FIG. 4C and 4D is depicted in FIG. 4E. As noted above, the frame rate improvement is determined by the total number of sub-regions (or individual detectors) included in the exposure sequence for the detector (or two-dimensional detector array). For example, in cases where a sequence of, for example, four exposures is used, corresponding to a 2×2 array of sub-regions or individual detectors, the resulting data acquisition frame rate may be four times that of the readout rate for the individual detector (or two-dimensional array of detectors). In some cases, the video data frame rate achieved using the disclosed methods and systems may be at least 4 times, at least 8 times, at least 12 times, at least 16 times, at least 20 times, at least 24 times, at least 28 times, at least 32 times, at least 64 times, at least 128 times, at least 256 times, at least 512 times, or higher. In some cases, the data acquisition frame rate achieved using the disclosed methods and systems may be any value within this range, for example, at least 144 times.

[0048] 4C illustrates the use of a two-dimensional deflector to deflect electrons into a grid of 3×3=9 subframes or exposure regions, but in some cases other patterns of exposure may also be used. In some cases, a two-dimensional deflector may be used to deflect electrons into a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20 subregions, or more. In some cases, a two-dimensional deflector may be used to deflect electrons into a non-square array of 3×4, 6×9, 9×12, 12×15, etc. subregions. In some cases, a two-dimensional deflector may be used to deflect electrons to a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20, or more different detectors in a two-dimensional detector array. In some cases, a two-dimensional deflector may be used to deflect electrons to a non-square array of individual detectors, such as 3×4, 6×9, 9×12, 12×15 detectors, etc.

[0049] In FIG. 4C, the subframes or sub-regions (i.e., exposure regions) are shown as contiguous rectangular regions. In some cases, they may be non-rectangular and / or non-contiguous. In some cases, the deflected beam may be projected through a circular aperture, thereby resulting in projecting an array of circular images onto the camera (see, e.g., FIG. 1). In some cases, there may be gaps between exposure regions consisting of pixels that are never exposed during a sequence. In some cases, there may be overlap between exposure regions such that a pixel is exposed multiple times before being read out.

[0050] In Figures 4C and 4D, the order of exposing different sub-frames / sub-regions of the camera (Figure 4C (spatial map) and Figure 4D (exposure timing diagram)) is shown in read order (see Figure 4A (spatial map) and Figure 4B (readout timing diagram)), however, the actual order may be any pattern that avoids exposing a specific region of the camera that is actively being read out. For example, in some cases a sinusoidal pattern may be used to achieve minimum switching times. In some cases, the order of exposure within each row of sub-regions may be shuffled. In some cases, the sub-frame exposure sequence may or may not be the same for every camera readout cycle. In some cases, each sub-frame may be exposed exactly once for a given camera readout cycle. In other cases, a sub-frame may be exposed multiple times. Such exposure sequences may or may not be used in implementing compressed sensing data acquisition modes. In some cases, some sub-frames may be skipped to create a reference mark in time and facilitate interpretation of the data.

[0051] The voltage waveform applied to each pair of deflector electrodes (X and Y pairs) is shown in FIG. 4E as having equal magnitude step changes. In some cases, one plate of each pair may be held at a fixed voltage, or the voltages applied to each pair may be different in magnitude or have the same or opposite signs. In some cases, the magnitude of the voltage waveform may range from about 0 volts to about 10 kV. In some cases, the magnitude of the voltage waveform may be at least 0 volts, at least 10 volts, at least 100 volts, at least 500 volts, at least 1,000 volts, at least 5 kV, or at least 10 kV. In some cases, the magnitude of the voltage waveform may be up to 10 kV, up to 5 kV, up to 1,000 volts, up to 500 volts, up to 100 volts, up to 10 volts, or about 0 volts. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the magnitude of the voltage waveform may range from about 100 volts to about 1,000 volts. In some cases, the magnitude of the voltage waveform may have any value within this range, for example, about 665 volts. The magnitude and timing sequence for the voltage waveform is programmable, comprising a sequence of four electrode voltages that determines the number of exposed regions, the order in which the sub-regions are exposed, and the duration of exposure in each sub-region (i.e., dwell time per sub-region).

[0052] Deflection of transmitted or scattered electrons between subframes / exposure regions may be accomplished in the systems illustrated in Figures 3 and 4A-4E using a sequence of voltages on electrode pairs that apply an electric field to the beam of electrons. In other implementations, the electrons may be deflected by using a magnetic field generated, for example, by a sequence of currents driven through magnetic coils. In still other implementations, there may be additional electrodes or electromagnets to which a voltage or current is applied such that image or diffraction data distortions introduced by the deflection process are reduced.

[0053] In some cases, the duration of exposure at each sub-region or individual detector in the two-dimensional detector array (i.e., the dwell time per sub-region or individual detector) may range from about 1 microsecond to about 100 milliseconds. In some cases, the dwell time may be at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, at least 1 millisecond, at least 10 milliseconds, or at least 100 milliseconds. In some cases, the dwell time may be up to 100 milliseconds, up to 10 milliseconds, up to 1 millisecond, up to 100 microseconds, up to 10 microseconds, or up to 1 microsecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, e.g., in some cases, the dwell time may range from about 10 microseconds to about 10 milliseconds. In some cases, the dwell time may have any value within this range, e.g., about 1.3 milliseconds. In some cases, the dwell times for each subframe / exposure region / individual detector may be equal to each other, as illustrated in the timing diagram of Figure 4D. In some cases, the dwell times for each subframe / exposure region / individual detector may differ from each other and may be individually adjustable.

[0054] In some cases, the methods and systems described herein are configured to provide high duty cycle operation without incurring time distortion of the data, hi some cases, the duty cycle may be at least 85%, at least 90%, at least 95%, at least 98%, or at least 99% without incurring time distortion of the data.

[0055] In general, the switching speed of the 2D deflector should be fast compared to the dwell time used for the exposure of the small area or individual detectors to minimize temporal distortion, i.e., so that there is no blurring between frames. In some cases, the switching time of the 2D deflector may range from about 1 nanosecond to about 1 millisecond. In some cases, the switching time is at least 1 nanosecond, at least 10 nanoseconds, at least 100 nanoseconds, at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, or at least 1 millisecond. In some cases, the switching time is at most 1 millisecond, at most 100 microseconds, at most 10 microseconds, at most 1 microsecond, at most 100 nanoseconds, at most 10 nanoseconds, or at most 1 nanosecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the switching time may range from about 10 nanoseconds to about 10 microseconds. In some cases, the switching times may have any value within this range, for example, about 1.3 milliseconds. In some cases, the switching times for switching between subframes / exposure regions / individual detectors may be equal to each other. In some cases, the switching times may be different and individually adjustable.

[0056] In general, the spatial positioning achieved by the two-dimensional deflector should be sufficiently accurate so that the jitter in repositioning the deflected image or diffraction pattern is less than about the dimension of a single image sensor pixel. In some cases, the spatial positioning achieved by the two-dimensional deflector is better than about 500 μm, better than about 450 μm, better than about 400 μm, better than about 350 μm, better than about 300 μm, better than about 250 μm, better than about 200 μm, better than about 150 μm, better than about 100 μm, better than about 90 μm, better than about 80 μm, better than about 70 μm. better than about 60 μm, better than about 50 μm, better than about 40 μm, better than about 30 μm, better than about 20 μm, better than about 10 μm, better than about 9 μm, better than about 8 μm, better than about 7 μm, better than about 6 μm, better than about 5 μm, better than about 4 μm, better than about 3 μm, better than about 2 μm, or better than about 1 μm.

[0057] In some cases, synchronization of the two-dimensional deflector with the detector readout cycle (e.g., a rolling shutter readout cycle for a CMOS camera) may be better than 50 ms, better than 40 ms, better than 30 ms, better than 20 ms, better than 10 ms, better than 9 ms, better than 8 ms, better than 7 ms, better than 6 ms, better than 5 ms, better than 4 ms, better than 3 ms, better than 2 ms, better than 1 ms, better than 900 microseconds, better than 800 microseconds, better than 700 microseconds, better than 600 microseconds, better than 500 microseconds, better than 400 microseconds, better than 300 microseconds, better than 200 microseconds, better than 100 microseconds, better than 90 microseconds, better than 10 ... The clock may be accurate to better than 40 microseconds, better than 10 microseconds, better than 80 microseconds, better than 70 microseconds, better than 60 microseconds, better than 50 microseconds, better than 40 microseconds, better than 30 microseconds, better than 20 microseconds, better than 10 microseconds, better than 9 microseconds, better than 8 microseconds, better than 7 microseconds, better than 6 microseconds, better than 5 microseconds, better than 4 microseconds, better than 3 microseconds, better than 2 microseconds, better than 1 microsecond, better than 900 nanoseconds, better than 800 nanoseconds, better than 700 nanoseconds, better than 600 nanoseconds, better than 500 nanoseconds, better than 400 nanoseconds, better than 300 nanoseconds, better than 200 nanoseconds, or better than 100 nanoseconds.

[0058] 4A and 4B illustrate a typical rolling shutter readout pattern in which the top row of pixels is read out, then the second row, then the third row, etc., down to the bottom row. However, the disclosed method applies to any continuous video readout performed to different regions of the camera (or different detectors in an array of detectors) at different times. For example, the readout may start with the left column of pixels and sweep across the camera to the right column of pixels. The readout may encompass multiple simultaneous rows or columns of pixels. The readout may be applied to a sequence of block regions of the camera's image sensor. The key is that exposure is directed to different regions of the camera / detector in a synchronized manner while they are not being read out, so that the continuous readout speed is achieved without the distortion associated with exposing regions during readout.

[0059] In some cases, the disclosed methods (and systems configured to implement them) may include a processor or computer-implemented algorithms and methods for processing the acquired data to generate high frame rate video data, hyperspectral image data, 2-D or N-D arrays of images, and / or diffraction pattern data, etc. For example, in some cases, the processor or computer-implemented algorithms and methods may (i) segment the raw camera readout data into sub-frames containing individual diffraction patterns or images, (ii) correct for distortions introduced by the deflection process, and (iii) align the sub-frames to then assemble an image stack, array of diffraction patterns, or video data that can be viewed directly or quantitatively analyzed according to the same methods that would have been used for slower data from the same detector in a standard operating mode. In some cases, these processing steps may utilize pre-processed calibration data from the same sample from which the image data or diffraction data was acquired. In some cases, these processing steps may utilize pre-processed calibration data for a different sample measured under similar conditions. In some cases, the processor or computer-implemented algorithms and methods may perform additional processing steps such as (i) fine-tuning the sub-frame alignment by identifying consistent features within the sub-frame data itself; (ii) using the fine-tuning of the sub-frame alignment to correct for sample drift during measurement; (iii) filtering and removing noise using prior knowledge of the sample or physical system being measured, the detector characteristics, and / or the optics itself; and / or (iv) inferring data from areas of the detector array that are exposed multiple times during a single readout cycle.

[0060] In some cases, the continuous high frame rate data compiled using the disclosed methods and systems may comprise from about 2 to about 4,194,304 (e.g., 2,048 x 2,048) frames of image or diffraction data. In some cases, the continuous high frame rate video data compiled using the disclosed methods and systems may comprise at least 2, at least 4, at least 8, at least 16, at least 32, at least 64, at least 128, at least 256, at least 512, at least 1,024, at least 2,048, at least 4,096, at least 8,192, at least 16,384, at least 32,768, at least 65,536, at least 131,072, at least 262,144, at least 524,288, at least 1,048,576, at least 2,097,152, or at least 4,194,304 frames of image or diffraction data. In some cases, the number of frames of the continuous high frame rate video data compiled using the disclosed methods and systems may have any value within this range, for example, about 6,200 frames of image or diffraction data. In some cases, the total time required to obtain data may range from about 1 second to about 8 hours, depending on the type of sample and / or measurements being made. In some cases, the total data acquisition time may be at least 1 second, at least 10 seconds, at least 30 seconds, at least 1 minute, at least 10 minutes, at least 30 minutes, at least 1 hour, at least 2 hours, at least 3 hours, at least 4 hours, at least 5 hours, at least 6 hours, at least 7 hours, or at least 8 hours.

[0061] High Frame Rate Optical Microscopy and Imaging: Although primarily discussed in the context of data capture for high frame rate transmission electron microscopy, the methods described above may also be applied to optical microscopy and imaging applications. For example, a high speed two-dimensional deflector suitable for deflecting photons rather than electrons may be used to deflect the pattern of photons transmitted, reflected, and / or scattered by a sample or object onto a series of two or more sub-regions of a two-dimensional detector (e.g., a CCD or CMOS image sensor) or a two-dimensional array of individual detectors. Examples of suitable two-dimensional deflectors for deflecting photons include, but are not limited to, rotating mirrors, rotating prisms, and the like. A CCD or CMOS camera or other detector designed and optimized for use with ultraviolet, visible, or infrared light, rather than one designed and optimized for the detection of electrons, would be selected.

[0062] With respect to the high frame rate transmission electron microscopy method described above, precise continuous synchronization of the 2D deflector modulation control signals with a continuous readout mode of the 2D detector, e.g., a rolling shutter mode for a CMOS camera, allows exposure of a sequence of two or more sub-regions of the detector that are not actively readout at the time of exposure. Processing of the image frames acquired by each of the sub-regions may then be used to assemble continuous video data having a frame rate significantly higher than the readout rate for the 2D detector.

[0063] Again, the frame rate improvement achieved is determined by the total number of sub-regions (or individual detectors) included in the exposure sequence / readout cycle for the detector (or two-dimensional detector array). For example, in cases where a sequence of, for example, four exposures is used, corresponding to a 2×2 array of sub-regions or individual detectors, the resulting data acquisition frame rate may be four times that of the readout rate for the individual detector (or two-dimensional array of detectors). In some cases, the video data frame rate achieved using the disclosed methods and systems may be at least four times, at least eight times, at least twelve times, at least sixteen times, at least twenty times, at least twenty-four times, at least twenty-eight times, at least thirty-two times, at least sixty-four times, at least twenty-eight ...

[0064] In some cases, a two-dimensional deflector may be used to deflect photons into a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20 subregions, or more. In some cases, a two-dimensional deflector may be used to deflect photons into a non-square array of 3×4, 6×9, 9×12, 12×15, etc. subregions. In some cases, a two-dimensional deflector may be used to deflect photons to a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20, or more different detectors in a two-dimensional detector array. In some cases, a two-dimensional deflector may be used to deflect photons to a non-square array of individual detectors, such as 3×4, 6×9, 9×12, 12×15 detectors, etc.

[0065] In some cases, the sub-frames or sub-regions (i.e., exposure regions) may be, for example, contiguous rectangular regions. In some cases, they may be non-rectangular and / or non-contiguous. In some cases, the deflected beam may be projected through a circular aperture, thereby resulting in projecting an array of circular images onto the camera. In some cases, there may be gaps between exposure regions consisting of pixels that are never exposed during a sequence. In some cases, there may be overlap between exposure regions such that a pixel is exposed multiple times before being read out.

[0066] In some cases, the order of exposing different subframes / sub-regions of the camera may be any pattern that avoids exposing a specific region of the camera that is actively being read out. For example, in some cases, a sinusoidal pattern may be used to achieve minimum switching times. In some cases, the order of exposure within each row of sub-regions may be shuffled. In some cases, the subframe exposure sequence may or may not be the same for every camera readout cycle. In some cases, each subframe may be exposed exactly once for a given camera readout cycle. In other cases, a subframe may be exposed multiple times. Such exposure sequences may or may not be used in implementing compressed sensing data acquisition modes. In some cases, some subframes may be skipped to create a reference mark in time and facilitate interpretation of the data.

[0067] In some cases, the duration of exposure in each sub-region or individual detector in the two-dimensional detector array (i.e., the dwell time per sub-region or individual detector) may range from about 1 microsecond to about 100 milliseconds. In some cases, the dwell time may be at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, at least 1 millisecond, at least 10 milliseconds, or at least 100 milliseconds. In some cases, the dwell time may be up to 100 milliseconds, up to 10 milliseconds, up to 1 millisecond, up to 100 microseconds, up to 10 microseconds, or up to 1 microsecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, e.g., in some cases, the dwell time may range from about 10 microseconds to about 10 milliseconds. In some cases, the dwell time may have any value within this range, e.g., about 1.3 milliseconds. In some cases, the dwell times per sub-frame / exposure region / individual detector may be equal to each other. In some cases, the dwell times for each subframe / exposure area / individual detector may be different from each other and may be individually adjustable.

[0068] In some cases, the methods and systems described herein are configured to provide high duty cycle operation without incurring time distortion of the data, hi some cases, the duty cycle may be at least 85%, at least 90%, at least 95%, at least 98%, or at least 99% without incurring time distortion of the data.

[0069] In general, the switching speed of the 2D deflector should be fast compared to the dwell time used for the exposure of the small area or individual detectors to minimize temporal distortion, i.e., so that there is no blurring between frames. In some cases, the switching time of the 2D deflector may range from about 1 nanosecond to about 1 millisecond. In some cases, the switching time is at least 1 nanosecond, at least 10 nanoseconds, at least 100 nanoseconds, at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, or at least 1 millisecond. In some cases, the switching time is at most 1 millisecond, at most 100 microseconds, at most 10 microseconds, at most 1 microsecond, at most 100 nanoseconds, at most 10 nanoseconds, or at most 1 nanosecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the switching time may range from about 10 nanoseconds to about 10 microseconds. In some cases, the switching times may have any value within this range, for example, about 1.3 milliseconds. In some cases, the switching times for switching between subframes / exposure regions / individual detectors may be equal to each other. In some cases, the switching times may be different and individually adjustable.

[0070] For electron microscopy, the spatial positioning achieved by the two-dimensional deflector should be sufficiently accurate so that jitter in repositioning the deflected image or diffraction pattern is less than about the dimension of a single image sensor pixel. In some cases, the spatial positioning achieved by the two-dimensional deflector is better than about 500 μm, better than about 450 μm, better than about 400 μm, better than about 350 μm, better than about 300 μm, better than about 250 μm, better than about 200 μm, better than about 150 μm, better than about 100 μm, better than about 90 μm, better than about 80 μm, better than about 70 μm. better than about 60 μm, better than about 50 μm, better than about 40 μm, better than about 30 μm, better than about 20 μm, better than about 10 μm, better than about 9 μm, better than about 8 μm, better than about 7 μm, better than about 6 μm, better than about 5 μm, better than about 4 μm, better than about 3 μm, better than about 2 μm, or better than about 1 μm.

[0071] Again, synchronization of the 2D deflector with the detector readout cycle (e.g., rolling shutter readout cycle for a CMOS camera) should be better than 50 ms, better than 40 ms, better than 30 ms, better than 20 ms, better than 10 ms, better than 9 ms, better than 8 ms, better than 7 ms, better than 6 ms, better than 5 ms, better than 4 ms, better than 3 ms, better than 2 ms, better than 1 ms, better than 900 microseconds, better than 800 microseconds, better than 700 microseconds, better than 600 microseconds, better than 500 microseconds, better than 400 microseconds, better than 300 microseconds, better than 200 microseconds, better than 100 microseconds, better than 90 microseconds, better than 10 ... In some embodiments, the time may be more accurate than 100 nanoseconds, better than 900 nanoseconds, better than 80 microseconds, better than 70 microseconds, better than 60 microseconds, better than 50 microseconds, better than 40 microseconds, better than 30 microseconds, better than 20 microseconds, better than 10 microseconds, better than 9 microseconds, better than 8 microseconds, better than 7 microseconds, better than 6 microseconds, better than 5 microseconds, better than 4 microseconds, better than 3 microseconds, better than 2 microseconds, better than 1 microsecond, better than 900 nanoseconds, better than 800 nanoseconds, better than 700 nanoseconds, better than 600 nanoseconds, better than 500 nanoseconds, better than 400 nanoseconds, better than 300 nanoseconds, better than 200 nanoseconds, or better than 100 nanoseconds.

[0072] As with the electron microscopy methods and systems discussed above, the disclosed optical methods (and optical systems configured to implement them) may further include a processor or computer-implemented algorithms and methods for processing the acquired data and generating high frame rate data. For example, in some cases, the processor or computer-implemented algorithms and methods may (i) segment the raw camera readout data into sub-frames containing individual images, (ii) correct for distortions introduced by the deflection process, and (iii) align the sub-frames to assemble an image stack or video data that can then be viewed directly or quantitatively analyzed according to the same methods that would have been used for slower data from the same detector in a standard operating mode. In some cases, these processing steps may utilize pre-processed calibration data from the same sample from which the image data was acquired. In some cases, these processing steps may utilize pre-processed calibration data for a different sample measured under similar conditions. In some cases, the processor or computer-implemented algorithms and methods may perform additional processing steps such as (i) fine-tuning the sub-frame alignment by identifying consistent features within the sub-frame data itself; (ii) using the fine-tuning of the sub-frame alignment to correct for sample drift during measurement; (iii) filtering and removing noise using prior knowledge of the sample or physical system being measured, the detector characteristics, and / or the optics itself; and / or (iv) inferring data from areas of the detector array that are exposed multiple times during a single readout cycle.

[0073] In some cases, the continuous high frame rate data compiled using the disclosed methods and systems may comprise from about 2 to about 4,194,304 (e.g., 2,048 x 2,048) frames of image or diffraction data. In some cases, the continuous high frame rate video data compiled using the disclosed methods and systems may comprise at least 2, at least 4, at least 8, at least 16, at least 32, at least 64, at least 128, at least 256, at least 512, at least 1,024, at least 2,048, at least 4,096, at least 8,192, at least 16,384, at least 32,768, at least 65,536, at least 131,072, at least 262,144, at least 524,288, at least 1,048,576, at least 2,097,152, or at least 4,194,304 frames of image or diffraction data. In some cases, the number of frames of the continuous high frame rate video data compiled using the disclosed methods and systems may have any value within this range, for example, about 6,200 frames of image or diffraction data. In some cases, the total time required to obtain data may range from about 1 second to about 8 hours, depending on the type of sample and / or measurements being made. In some cases, the total data acquisition time may be at least 1 second, at least 10 seconds, at least 30 seconds, at least 1 minute, at least 10 minutes, at least 30 minutes, at least 1 hour, at least 2 hours, at least 3 hours, at least 4 hours, at least 5 hours, at least 6 hours, at least 7 hours, or at least 8 hours.

[0074] In some cases, the disclosed methods for high frame rate acquisition of optical image data may be implemented in the context of optical microscopy for imaging cells or other microscopic structures. In some cases, the disclosed methods for high frame rate acquisition of optical image data may be implemented in the context of high speed video cameras for imaging moving objects.

[0075] High Dynamic Range Electron Microscopy: Measurement of diffraction data using an electron microscope (EM) is an essential tool for material characterization and analysis. Examples include, but are not limited to, selected area electron diffraction (SAED), convergent beam electron diffraction (CBED), and transmission Kikuchi diffraction (TKD) measurements performed in a transmission electron microscope (TEM). One of the major challenges for collecting diffraction data is that the data typically exceeds the dynamic range of detectors currently on the market. This leads to detector saturation effects that can distort or obscure important information (Figure 5). The ability to more faithfully capture the full dynamic range of diffraction pattern data (or image data) at high throughput without incurring detector saturation issues as imparted by the methods described herein constitutes a significant advancement over the state of the art.

[0076] Other techniques have been used in past attempts to address the problem of limited dynamic range in image-type data, but not all of these techniques have been applied to diffraction data or even electron microscopy. Selective occlusion has become standard practice for many TEM diffraction data acquisition systems. This approach involves simply placing an object such as a needle in front of the detector at a predictable location of the bright central spot that corresponds to the impact point for non-diffracted electrons (see FIG. 5). This process sacrifices some of the diffraction pattern data, allowing for the measurement of dimmer diffracted spots further from the center. Unfortunately, it is generally not possible to block all of the non-diffracted electrons and completely eliminate detector saturation, resulting in a significant loss of useful information. The methods disclosed herein allow for the capture of information from both dim and bright spots across the entire image area.

[0077] Another prior art method is high speed movie / dose splitting, where video is captured at a high frame rate and an arbitrary number of frames are summed to obtain the desired exposure time. While the method disclosed herein uses electrostatic deflectors, which allow control of exposure over a much wider range of values ​​(to date, we have demonstrated exposure times ranging from 1 second to 50 nanoseconds), camera full frame exposures are typically 1 millisecond or longer even for the most advanced cameras. Furthermore, the disclosed method provides faster total acquisition times; for example, in situations where cycle times are limited by detector readout latency, the disclosed system can expose one detector while another is being read out, and data from multiple exposure times can be captured and transferred in the equivalent of a single detector readout cycle. This speed is important in applications such as 4D STEM, which can be used to acquire images at many locations on a sample, typically limited by sensor throughput.

[0078] Dual EEL (see International Publication No. WO2019 / 028129A1) is another related technique, but quite distinct from the disclosed method. Electron energy loss spectroscopy (EELS) spectra are measured on a special detector that converts energy information into spatial shifts on the camera. A second shifted copy of the spectrum is obtained on the same detector at a different exposure time, and the two copies can be analyzed separately to extract more information from the same measurement. The exposure times for the two copies of the spectrum can be varied independently. The implementation uses a high-speed one-dimensional electrostatic deflector to direct electrons to different areas of a CCD or CMOS camera. While the disclosed method is directed to improving the dynamic range of image and diffraction data acquisition, dual EELS is a method used to perform electron energy spectroscopy in a system with a single detector. Furthermore, the disclosed method allows for modulating the magnification and position of the image or diffraction data on the camera sensor (or on an array of different sensors), which the dual EELS approach does not.

[0079] Direct detection EELS (U.S. Patent Application Publication No. 2017 / 0207059A1, hereafter "the '059 patent application") is an approach in which a counting mode high speed direct electron detector is integrated into an EEL spectrometer and a high speed electrostatic beam blanker controls the exposure time. Multiple detector readouts with different exposure times are processed sequentially to generate a single data set with optimal dynamic range. Counting mode direct detection is a new method of imaging electrons that exhibits lower noise in some circumstances. The approach disclosed in the '059 patent application aims to overcome the drawbacks of counting mode, i.e., very low maximum electron dose rate and therefore dynamic range, but differs from the method disclosed herein. While the disclosed method uses electrostatic or electromagnetic optics to switch image or diffraction data to different detectors or different pixel areas within a single detector, the '059 patent application uses electrostatic optics to block electrons from reaching the camera. The disclosed methods and systems allow for data to be collected with potentially higher duty cycles and reduced doses to the sample.

[0080] A method frequently used in image capture by cell phone cameras and other scientific imaging applications involves taking two photographs at different exposure times and post-processing the images to produce a single high dynamic range image. Drift in sample alignment and other imaging conditions during measurements makes this method less reliable, and its use is being phased out from top-of-the-line cell phone cameras in favor of other techniques, such as the Sony Interline HDR approach, where the image sensor comprises alternating rows of pixels that are read out at different exposure levels and stitched together into a single image in post-processing. This approach offers an improvement over previous cell phone methods, as it allows for the acquisition of HDR images using one exposure instead of two. Newer sensors, such as the Sony IMX294CJK color SMOS image sensor, have more complex pixel patterns that use different simultaneous exposure levels. These approaches differ from the disclosed methods and systems in that, in some cases, the disclosed methods and systems for improving the dynamic range of image and diffraction data acquisition may include the use of multiple detectors. Furthermore, the disclosed method and system allows for modulating the image or diffraction data on the camera sensor and / or the magnification and position of the probe beam on the sample.The use of high-speed two-dimensional deflectors in the disclosed method and system also provides better control over exposure time.Finally, since the inter-row HDR approach does not have data available for the same position in the image at multiple exposure levels, inter-row HDR relies more heavily on interference, which may not be desirable in scientific applications.

[0081] In contrast to the methods and systems described herein, none of the prior art methods utilize rapid deflection of electrons transmitted or scattered by a sample between multiple distinct detectors (or in some cases, different sub-regions of a single detector) that can be individually adjusted for sensitivity and / or saturation levels. Individual image frames captured by the different detectors (or sub-regions of a single detector) may then be processed to generate one or more image frames with a much higher dynamic range than is achievable with a single image sensor or a single full-frame image sensor readout. The disclosed method offers a higher dynamic range than previous methods, increased versatility in terms of detector types, minimal complexity, and more control over exposure levels.

[0082] In TEM image detectors, there is a trade-off between sensitivity and resistance to damage. In some situations, it may be advantageous to alternate between arrays of detectors that are independently optimized for different measurements and situations. For example, the electron beam may briefly visit the sensor and measure the current or verify proper alignment during a measurement sequence. Or a relatively long exposure time (i.e., the dwell time of the beam on a given detector or detector sub-region) may be given to a specific region of the camera or detector array that is relatively more resistant to electron radiation damage. In applications where the experimental parameters are modulated in concert with the video measurement, there may be times when the data is temporarily invalid or noisy. To prevent contamination of the broader data set, the image data can be invalidated or deflected to a detector region that has zero sensitivity during such times. Due to the wide variety of potential applications, the sensor or detector used in the disclosed methods and systems may in some cases simply be an unmodified commercial TEM image detector, or in other cases may be an array of detectors optimized for different performance characteristics or purposes. Diffraction pattern measurements are one of the EM applications most limited by dynamic range. However, the disclosed methods are equally applicable to real space TEM imaging applications.

[0083] As noted, the disclosed methods and systems improve the dynamic range of measurements in an electron microscope (or in some cases, an optical system, as will be discussed further below) by rapidly deflecting an electron beam to expose multiple detectors (or different sub-regions within a single detector) to the deflected electrons for different precisely controlled exposure times. In some cases, precise continuous synchronization of the 2D deflector modulation control signal with a continuous readout mode of the 2D detector, e.g., a rolling shutter mode for a CMOS camera, allows exposure of a series of two or more sub-regions of the detector that are not actively readout at the time of exposure, and further processing of the image frames acquired by each of the sub-regions or individual detectors may then be used to assemble continuous high dynamic range video data with a frame rate significantly higher than the readout rate for the CMOS camera or 2D detector array.

[0084] In some cases, the detectors in the two-dimensional array of detectors may be identical copies of a particular sensor, or copies of a particular sensor individually modified using appropriate filters, masks, or sensitivity or saturation level adjustments for different electron exposure levels. As noted, in some cases, the detectors may be sub-areas of a single compound detector, such as a CMOS or CCD camera. In some cases, the detector array may contain a combination of different devices optimized to sense, for example, images, electron beam current, electron beam alignment, or other diagnostic quantities. Examples of suitable devices include, but are not limited to, CCD cameras, CMOS cameras, dark-field STEM detectors, Faraday cups, quad photodiodes, annular dark-field detectors, bright-field detectors, general-purpose detectors, or any combination thereof. In some cases, the sensitivity of a given detector may be modified in a spatially varying pattern, for example, to block the bright central spot where undiffracted electrons typically strike the camera in some diffraction measurements.

[0085] In some cases, the HDR systems disclosed herein may be configured to modulate the magnification and position of the deflected electron pattern on the camera and / or the probe beam on the sample. In some cases, the magnification of the deflected electron pattern and / or the probe beam may be at least 2x, 4x, 8x, 16x, or 32x that of the unmagnified deflected electron pattern or probe beam.

[0086] In one non-limiting example of implementation, exposure may be controlled by a high-speed electrostatic deflector placed between the sample and the detector array, as illustrated in FIG. 6. The deflector applies an electric field and steers the image of the electron beam or diffraction pattern containing the image data onto a defined detector. The deflector may steer the beam in one or two spatial dimensions, depending on the layout of the detector array. The dwell time sets the exposure level of each detector. For example, in a four detector array, detector 1 may be exposed for 100 nanoseconds, then detector 2 may be exposed for 1 microsecond, then detector 3 may be exposed for 10 microseconds, then detector 4 may be exposed for 100 microseconds. After exposure is completed, the deflector may steer the beam away from all the detectors. In some cases, the deflector may also adjust the magnification of the image to compensate for varying detector heights and positions, or vary the magnification of the resulting data.

[0087] The deflector is synchronized to the readout cycle of the detector array so that readout does not conflict with exposure. Timing control of other experimental parameters, such as scanning control of the incident beam position on the sample (e.g. in STEM measurements), focusing of the beam on the sample, pre-sample beam blankers to control sample exposure, or manipulation of the sample itself, may also be integrated with the deflector system. Each detector may be exposed and readout once or more during a single measurement sequence. A flow chart for the complete measurement process is illustrated in FIG. 7. As shown in Fig. 7, the measurement process may include (i) deflecting the electron beam to one or more detectors used to check alignment and / or perform other beam diagnostics, (ii) performing alignment and / or beam diagnostics, (iii) optionally modulating one or more experimental parameters that affect the measurement process (e.g., changing the tilt angle of the sample relative to the electron beam), (iv) deflecting the electron beam to one or more detectors or sub-regions of the detector, where the dwell time (exposure) can be varied for different detectors or sub-regions, and (v) reading out one or more detectors or sub-regions of the detector. The process illustrated in Fig. 7 is also applicable to optical implementations of the disclosed HDR methods and systems, as will be discussed further below.

[0088] The disclosed methods may also include processor or computer implemented algorithms and methods for reconstructing a single measurement (e.g., a single frame of image or diffraction data) from data readouts from multiple detectors (FIG. 8). The detector readouts may be processed by a processor or computer for analysis. Analysis methods may include correction for distortions, including those introduced by deflectors, calibration of the incoming data incorporating known exposure times and characteristics of each detector, alignment and registration of the data such that different detector readouts may be spatially overlapped, and performing a combination of the individual measurements, appropriately weighting areas that are over- or under-exposed on a given detector, such that the combined data set contains an optimal combination of the individual data sets and accurately represents the quantity being measured. The resulting combined data set has a substantially improved dynamic range.

[0089] FIG. 9 provides a schematic diagram of a scanning transmission electron microscope (STEM) scan controller (scanning an electron beam probe across the sample plane) combined with a two-dimensional electrostatic deflector to deflect electrons transmitted or scattered by the sample to different detectors or sub-areas of the detector. In some cases, the disclosed methods may also be implemented in conventional transmission electron microscopy, i.e., in an operational mode that does not scan an electron beam probe across the sample plane. FIGS. 10A-E provide non-limiting examples of timing diagrams for controlling the electrostatic deflection of electrons in a TEM to expose different sub-areas of a two-dimensional detector or different detectors in a two-dimensional array of detectors. The system hardware used to implement the disclosed high dynamic range data acquisition methods is the same or similar to that used to implement the high frame rate data acquisition methods described above, e.g., high speed two-dimensional electrostatic or electromagnetic deflectors with fast switching times and precise positioning. The relevant time series for subframes / exposure areas / detector dwell times are also the same. In some cases, the present disclosure includes systems that combine HDR and HFR operations as described above, such that the exposure area of ​​the camera is synchronized to the readout pattern for the detector or detector array.

[0090] The two-dimensional electrostatic deflector system, diagrammatically illustrated in FIG. 9, is shown synchronized to a STEM scan controller that directs the electron beam probe to a sequence of sample locations (typically a square array of 128×128 pixels to 4,096×4,096 pixels with uniform dwell time). However, in other implementations, the probe pattern may be some arbitrary pattern of locations and / or exposure times on the sample. In some cases, the subframe / exposure region / detector deflection sequence may be synchronized to modulation of other parameters of the probe beam beyond its position on the sample, such as focal depth or convergence angle. In some cases, timing synchronization between the probe and subframe / exposure region / detector deflection sequences may be accomplished by sending signals from the STEM controller to the deflector controller or vice versa. In some cases, the signals may correspond to a signal for every STEM probe location, each line in a STEM probe scan, or some initial signal indicating the start of a preprogrammed sequence.

[0091] In some cases, timing synchronization between the camera / detector readout and the subframe / exposure area / detector deflection sequence can be established by control signals sent in either direction between the deflector control system and the camera / detector. In some cases, the number of camera subframes / exposure areas / detectors exposed per probe location on the sample (or other combination of probe parameters) can vary. In some cases, the number of camera subframes / exposure areas / detectors exposed per probe location on the sample (or other combination of probe parameters) can be the same.

[0092] As noted, Figures 10A-E provide non-limiting examples of timing diagrams for controlling electrostatic deflection of electrons in a TEM and exposing different sub-regions of a two-dimensional detector or different detectors in a two-dimensional array of detectors. Figure 10A provides a schematic diagram of a sequence of probe beam locations on a sample. A corresponding timing diagram for scanning the probe beam across the sample is depicted in Figure 10B. Figure 10C illustrates an array of ten detectors or detection regions. In some cases, for example, detection regions 1-9 may be sub-regions of a single detector, e.g., a CMOS camera, and detection region 10 may comprise a separate detector, e.g., a beam diagnostic sensor. Figure 10D illustrates a timing diagram for implementing the sequence of exposures illustrated in Figure 10C, with different exposure times (dwell times) for different detectors / detector regions. Figure 10E illustrates a timing diagram of voltages applied between electrode pairs in a two-dimensional electrostatic deflector to implement the deflection patterns and timing depicted in Figures 10C and 10D.

[0093] In some cases, for example, the number of camera subframes / exposure areas / detectors exposed per probe location on a sample (or other combination of probe parameters) may range from about 2 to about 20. In some cases, the number of camera subframes / exposure areas / detectors exposed per probe location on a sample (or other combination of probe parameters) may be at least 2, at least 3, at least 4, at least 5, at least 6, at least 7, at least 8, at least 9, at least 10, at least 15, or at least 20. In some cases, the number of camera subframes / exposure areas / detectors exposed per probe location on a sample (or other combination of probe parameters) may be up to 20, up to 15, up to 10, up to 9, up to 8, up to 7, up to 6, up to 5, up to 4, up to 3, or up to 2. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the number of camera sub-frames / exposure areas / detectors exposed per probe location on the sample (or other combination of probe parameters) may range from about 4 to about 15. In some cases, the number of camera sub-frames / exposure areas / detectors exposed per probe location on the sample (or other combination of probe parameters) may be any value within this range, for example, about 12. In some cases, the different exposure areas / detectors (e.g., detector areas 1-10 in FIG. 9) may comprise different detectors. In some cases, all or some of the different exposure areas / detectors (e.g., detector areas 1-9 in FIG. 9) may comprise sub-areas of a single detector.

[0094] In some cases, the improvement in dynamic range achieved using the disclosed methods and systems can be at least 2-fold, at least 4-fold, at least 8-fold, at least 12-fold, at least 16-fold, at least 20-fold, at least 40-fold, at least 60-fold, at least 80-fold, at least 100-fold, at least 250-fold, at least 500-fold, at least 750-fold, at least 1,000-fold, at least 2,000-fold, at least 3,000-fold, at least 4,000-fold, at least 5,000-fold, or higher. In some cases, the improvement in dynamic range achieved can be any value within this range, for example, at least 15-fold.

[0095] For the high frame rate data acquisition methods described above, different patterns of exposure may be used in acquiring the HDR data set. In some cases, a two-dimensional deflector may be used to deflect electrons to, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20 different sub-areas or different detectors, or even more patterns. In some cases, a two-dimensional deflector may be used to deflect electrons to a non-square array of 3×4, 6×9, 9×12, 12×15, etc. sub-areas or detectors.

[0096] In some cases, the dwell times or duration of exposure for a given probe setting in a given subframe / sub-region / exposure region / detector may all be the same, or they may be different. In some cases, the duration of exposure in each sub-region or individual detector in the two-dimensional detector array (i.e., the dwell time per sub-region or individual detector) may range from about 1 microsecond to about 100 milliseconds. In some cases, the dwell time may be at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, at least 1 millisecond, at least 10 milliseconds, or at least 100 milliseconds. In some cases, the dwell time may be up to 100 milliseconds, up to 10 milliseconds, up to 1 millisecond, up to 100 microseconds, up to 10 microseconds, or up to 1 microsecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the dwell time may range from about 10 microseconds to about 10 milliseconds. In some cases, the residence time may have any value within this range, for example, about 1.3 milliseconds.

[0097] In some cases, the switching time of the two-dimensional deflector may range from about 1 nanosecond to about 1 millisecond. In some cases, the switching time is at least 1 nanosecond, at least 10 nanoseconds, at least 100 nanoseconds, at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, or at least 1 millisecond. In some cases, the switching time is at most 1 millisecond, at most 100 microseconds, at most 10 microseconds, at most 1 microsecond, at most 100 nanoseconds, at most 10 nanoseconds, or at most 1 nanosecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the switching time may range from about 10 nanoseconds to about 10 microseconds. In some cases, the switching time may have any value within this range, for example, about 1.3 milliseconds. In some cases, the switching times for switching between subframes / exposure regions / individual detectors may be equal to each other. In some cases, the switching times may be different and individually adjustable.

[0098] The voltage waveforms applied to each pair of deflector electrodes (X and Y pairs) depicted in FIG. 9 may comprise step changes of equal or different magnitudes. In some cases, one plate of each pair may be held at a fixed voltage, or the voltages applied to each pair may be of different magnitudes or have the same or opposite signs. In some cases, the magnitude of the voltage waveform may range from about 0 volts to about 10 kV. In some cases, the magnitude of the voltage waveform may be at least 0 volts, at least 10 volts, at least 100 volts, at least 500 volts, at least 1,000 volts, at least 5 kV, or at least 10 kV. In some cases, the magnitude of the voltage waveform may be up to 10 kV, up to 5 kV, up to 1,000 volts, up to 500 volts, up to 100 volts, up to 10 volts, or about 0 volts. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the magnitude of the voltage waveform may range from about 100 volts to about 1,000 volts. In some cases, the magnitude of the voltage waveform may have any value within this range, for example, about 1,250 volts. The magnitude and timing sequence for the voltage waveform is programmable, comprising a sequence of four electrode voltages that determines the number of exposed regions, the order in which the subregions are exposed, and the duration of exposure in each subregion (i.e., the dwell time per subregion within an individual detector).

[0099] Deflection of transmitted or scattered electrons between subframes / exposure regions may be accomplished in the systems illustrated in Figures 9 and 10A-10E using a sequence of voltages on electrode pairs that apply an electric field to the beam of electrons. In other implementations, the electrons may be deflected by using a magnetic field generated, for example, by a sequence of currents driven through magnetic coils. In still other implementations, there may be additional electrodes or electromagnets to which a voltage or current is applied such that image or diffraction data distortions introduced by the deflection process are reduced.

[0100] For the high frame rate data acquisition methods and systems described above, the spatial positioning achieved by the two-dimensional deflector should be sufficiently accurate so that jitter in repositioning the deflected image or diffraction pattern is less than about the dimension of a single image sensor pixel. In some cases, the spatial positioning achieved by the two-dimensional deflector is better than about 500 μm, better than about 450 μm, better than about 400 μm, better than about 350 μm, better than about 300 μm, better than about 250 μm, better than about 200 μm, better than about 150 μm, better than about 100 μm, better than about 90 μm, better than about 80 μm, better than about 70 μm. better than about 60 μm, better than about 50 μm, better than about 40 μm, better than about 30 μm, better than about 20 μm, better than about 10 μm, better than about 9 μm, better than about 8 μm, better than about 7 μm, better than about 6 μm, better than about 5 μm, better than about 4 μm, better than about 3 μm, better than about 2 μm, or better than about 1 μm.

[0101] In some cases, synchronization of the two-dimensional deflector with the detector readout cycle (e.g., a rolling shutter readout cycle for a CMOS camera) may be better than 50 ms, better than 40 ms, better than 30 ms, better than 20 ms, better than 10 ms, better than 9 ms, better than 8 ms, better than 7 ms, better than 6 ms, better than 5 ms, better than 4 ms, better than 3 ms, better than 2 ms, better than 1 ms, better than 900 microseconds, better than 800 microseconds, better than 700 microseconds, better than 600 microseconds, better than 500 microseconds, better than 400 microseconds, better than 300 microseconds, better than 200 microseconds, better than 100 microseconds, better than 90 microseconds, better than 10 ... The clock may be accurate to better than 40 microseconds, better than 10 microseconds, better than 80 microseconds, better than 70 microseconds, better than 60 microseconds, better than 50 microseconds, better than 40 microseconds, better than 30 microseconds, better than 20 microseconds, better than 10 microseconds, better than 9 microseconds, better than 8 microseconds, better than 7 microseconds, better than 6 microseconds, better than 5 microseconds, better than 4 microseconds, better than 3 microseconds, better than 2 microseconds, better than 1 microsecond, better than 900 nanoseconds, better than 800 nanoseconds, better than 700 nanoseconds, better than 600 nanoseconds, better than 500 nanoseconds, better than 400 nanoseconds, better than 300 nanoseconds, better than 200 nanoseconds, or better than 100 nanoseconds.

[0102] In some cases, the camera / detector exposure sequence for a given set of probe parameters (e.g., position on the sample) may be different from that used for a different set of probe parameters. In some cases, during transitions between certain sets of probe parameters, the diffraction data may be smeared or invalid. In such cases, deflectors may be used to direct the data to areas of the detector where it would be ignored, or where no detector is located.

[0103] In some cases, the disclosed methods (and systems configured to implement them) may include a processor or computer-implemented algorithms and methods for processing the acquired data and generating high dynamic range image or video data. For example, in some cases, the processor or computer-implemented algorithms and methods may (i) segment the raw camera readout data into subframes containing individual diffraction patterns or images, (ii) correct for distortions introduced by the deflection process, and (iii) combine related series / groups of subframes together, each series containing data from two or more subframes having the same measurement parameters except for their exposure times, to generate individual patterns or images with improved dynamic range. In some cases, these processing steps may utilize pre-processed calibration data from the same sample from which the image data or diffraction data was acquired. In some cases, these processing steps may utilize pre-processed calibration data for different samples measured under similar conditions. In some cases, the processor or computer-implemented algorithms and methods may perform additional processing steps such as (i) combining different detection channels (e.g., data sets from different types of detectors) to generate a multi-modal data set; (ii) fine-tuning the sub-frame alignment by identifying consistent features within the sub-frame data itself; (iii) using the fine-tuning of the sub-frame alignment to correct for sample drift during measurement; (iv) filtering and removing noise using prior knowledge of the sample or physical system being measured, the characteristics of the detector, and / or the optical system itself; and / or (v) inferring data from areas of the detector array that are exposed multiple times during a single readout cycle.

[0104] High Dynamic Range Optical Microscopy and Imaging: Although primarily discussed in the context of data capture for high dynamic range transmission electron microscopy, the methods described above may also be applied to optical microscopy and imaging applications. For example, a high-speed two-dimensional deflector suitable for deflecting photons rather than electrons may be used to deflect the pattern of photons transmitted, reflected, and / or scattered by a sample or object onto a series of two or more sub-regions of a two-dimensional detector (e.g., a CCD or CMOS image sensor) or a two-dimensional array of individual detectors. Examples of suitable two-dimensional deflectors for deflecting photons include, but are not limited to, rotating mirrors, rotating prisms, and the like. A CCD or CMOS camera or other detector designed and optimized for use with ultraviolet, visible, or infrared light, rather than one designed and optimized for the detection of electrons, would be selected.

[0105] With respect to the high dynamic range transmission electron microscopy method described above, precise adjustment of the dwell time (exposure time) for each of the sub-regions or individual detectors, and / or adjustment of the sensitivity and / or saturation level for each of the sub-regions or individual detectors allows for the acquisition of a series of images that can be processed to generate a single image frame with a higher dynamic range than is achievable using a single detector. In some cases, precise continuous synchronization of the two-dimensional deflector modulation control signal with the continuous readout mode of the two-dimensional detector, e.g., rolling shutter mode for a CMOS camera, allows for the exposure of a series of two or more sub-regions of the detector that are not actively readout at the time of exposure, and further, processing of the image frames acquired by each of the sub-regions or individual detectors may then be used to assemble a continuous high dynamic range video data with a frame rate significantly higher than the readout rate for the CMOS camera or two-dimensional detector array. In some cases, the HDR system disclosed herein may be configured to modulate the magnification as well as the position of the deflected photon pattern on the camera. In some cases, the magnification of the deflected electron pattern may be at least 2, 4, 8, 16, or 32 times that of the unmagnified deflected electron pattern.

[0106] With respect to the high frame rate transmission electron microscopy methods described above, the number of camera sub-frames / exposure areas / detectors exposed by the deflection of photons transmitted, reflected, or scattered by a sample or object within a readout cycle for a single detector or array of detectors may range from about 2 to about 20. In some cases, the number of exposed camera sub-frames / exposure areas / detectors may be at least 2, at least 3, at least 4, at least 5, at least 6, at least 7, at least 8, at least 9, at least 10, at least 15, or at least 20. In some cases, the number of exposed camera sub-frames / exposure areas / detectors may be up to 20, up to 15, up to 10, up to 9, up to 8, up to 7, up to 6, up to 5, up to 4, up to 3, or up to 2. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the number of exposed camera sub-frames / exposure areas / detectors may range from about 4 to about 15. In some cases, the number of exposed camera sub-frames / exposure areas / detectors may be any value within this range, for example, about 14.

[0107] In some cases, the improvement in dynamic range achieved using the disclosed methods and systems can be at least 2-fold, at least 4-fold, at least 8-fold, at least 12-fold, at least 16-fold, at least 20-fold, at least 40-fold, at least 60-fold, at least 80-fold, at least 100-fold, at least 250-fold, at least 500-fold, at least 750-fold, at least 1,000-fold, at least 2,000-fold, at least 3,000-fold, at least 4,000-fold, at least 5,000-fold, or higher. In some cases, the improvement in dynamic range achieved can be any value within this range, for example, at least 17-fold.

[0108] In some cases, the optical HDR systems disclosed herein may be configured to modulate the magnification and position of the deflected photons or image on a camera or detector array. In some cases, the magnification of the deflected photons (e.g., diffraction pattern or image) may be at least 2x, 4x, 8x, 16x, or 32x that of the unmagnified deflected diffraction pattern or image.

[0109] In some cases, a two-dimensional deflector may be used to deflect photons into a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20 subregions, or more. In some cases, a two-dimensional deflector may be used to deflect photons into a non-square array of 3×4, 6×9, 9×12, 12×15, etc. subregions. In some cases, a two-dimensional deflector may be used to deflect photons to a pattern of, for example, 2×2, 3×3, 4×4, 5×5, 6×6, 7×7, 8×8, 9×9, 10×10, 11×11, 12×12, 13×13, 14×14, 15×15, 16×16, 17×17, 18×18, 19×19, 20×20, or more different detectors in a two-dimensional detector array. In some cases, a two-dimensional deflector may be used to deflect photons to a non-square array of individual detectors, such as 3×4, 6×9, 9×12, 12×15 detectors, etc.

[0110] In some cases, the sub-frames or sub-regions (i.e., exposure regions) may be, for example, contiguous rectangular regions. In some cases, they may be non-rectangular and / or non-contiguous. In some cases, the deflected beam may be projected through a circular aperture, thereby resulting in projecting an array of circular images onto the camera. In some cases, there may be gaps between exposure regions consisting of pixels that are never exposed during a sequence. In some cases, there may be overlap between exposure regions such that a pixel is exposed multiple times before being read out.

[0111] In some cases, the order of exposing different subframes / sub-regions of the camera may be any pattern that avoids exposing a specific region of the camera that is actively being read out. For example, in some cases, a sinusoidal pattern may be used to achieve minimum switching times. In some cases, the order of exposure within each row of sub-regions may be shuffled. In some cases, the subframe exposure sequence may or may not be the same for every camera readout cycle. In some cases, each subframe may be exposed exactly once for a given camera readout cycle. In other cases, a subframe may be exposed multiple times. Such exposure sequences may or may not be used in implementing compressed sensing data acquisition modes. In some cases, some subframes may be skipped to create a reference mark in time and facilitate interpretation of the data.

[0112] In some cases, the duration of exposure in each sub-region or individual detector in the two-dimensional detector array (i.e., the dwell time per sub-region or individual detector) may range from about 1 microsecond to about 100 milliseconds. In some cases, the dwell time may be at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, at least 1 millisecond, at least 10 milliseconds, or at least 100 milliseconds. In some cases, the dwell time may be up to 100 milliseconds, up to 10 milliseconds, up to 1 millisecond, up to 100 microseconds, up to 10 microseconds, or up to 1 microsecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, e.g., in some cases, the dwell time may range from about 10 microseconds to about 10 milliseconds. In some cases, the dwell time may have any value within this range, e.g., about 1.3 milliseconds. In some cases, the dwell times per sub-frame / exposure region / individual detector may be equal to each other. In some cases, the dwell times for each subframe / exposure area / individual detector may be different from each other and may be individually adjustable.

[0113] In some cases, the methods and systems described herein are configured to provide high duty cycle operation without incurring time distortion of the data, hi some cases, the duty cycle may be at least 85%, at least 90%, at least 95%, at least 98%, or at least 99% without incurring time distortion of the data.

[0114] In general, the switching speed of the 2D deflector should be fast compared to the dwell time used for the exposure of the small area or individual detectors to minimize temporal distortion, i.e., so that there is no blurring between frames. In some cases, the switching time of the 2D deflector may range from about 1 nanosecond to about 1 millisecond. In some cases, the switching time is at least 1 nanosecond, at least 10 nanoseconds, at least 100 nanoseconds, at least 1 microsecond, at least 10 microseconds, at least 100 microseconds, or at least 1 millisecond. In some cases, the switching time is at most 1 millisecond, at most 100 microseconds, at most 10 microseconds, at most 1 microsecond, at most 100 nanoseconds, at most 10 nanoseconds, or at most 1 nanosecond. Any of the lower and upper limits described in this paragraph may be combined to form a range included within the present disclosure, for example, in some cases, the switching time may range from about 10 nanoseconds to about 10 microseconds. In some cases, the switching times may have any value within this range, for example, about 1.3 milliseconds. In some cases, the switching times for switching between subframes / exposure regions / individual detectors may be equal to each other. In some cases, the switching times may be different and individually adjustable.

[0115] In some cases, the spatial positioning achieved by the two-dimensional optical deflector should be sufficiently accurate so that jitter in repositioning the deflected image is less than about the dimension of a single image sensor pixel. In some cases, the spatial positioning achieved by the two-dimensional optical deflector is better than about 500 μm, better than about 450 μm, better than about 400 μm, better than about 350 μm, better than about 300 μm, better than about 250 μm, better than about 200 μm, better than about 150 μm, better than about 100 μm, better than about 90 μm, better than about 80 μm, better than about 70 μm. better than about 60 μm, better than about 50 μm, better than about 40 μm, better than about 30 μm, better than about 20 μm, better than about 10 μm, better than about 9 μm, better than about 8 μm, better than about 7 μm, better than about 6 μm, better than about 5 μm, better than about 4 μm, better than about 3 μm, better than about 2 μm, or better than about 1 μm.

[0116] In some cases, synchronization of the two-dimensional deflector with the detector readout cycle (e.g., a rolling shutter readout cycle for a CMOS camera) may be better than 50 ms, better than 40 ms, better than 30 ms, better than 20 ms, better than 10 ms, better than 9 ms, better than 8 ms, better than 7 ms, better than 6 ms, better than 5 ms, better than 4 ms, better than 3 ms, better than 2 ms, better than 1 ms, better than 900 microseconds, better than 800 microseconds, better than 700 microseconds, better than 600 microseconds, better than 500 microseconds, better than 400 microseconds, better than 300 microseconds, better than 200 microseconds, better than 100 microseconds, better than 90 microseconds, better than 10 ... The clock may be accurate to better than 40 microseconds, better than 10 microseconds, better than 80 microseconds, better than 70 microseconds, better than 60 microseconds, better than 50 microseconds, better than 40 microseconds, better than 30 microseconds, better than 20 microseconds, better than 10 microseconds, better than 9 microseconds, better than 8 microseconds, better than 7 microseconds, better than 6 microseconds, better than 5 microseconds, better than 4 microseconds, better than 3 microseconds, better than 2 microseconds, better than 1 microsecond, better than 900 nanoseconds, better than 800 nanoseconds, better than 700 nanoseconds, better than 600 nanoseconds, better than 500 nanoseconds, better than 400 nanoseconds, better than 300 nanoseconds, better than 200 nanoseconds, or better than 100 nanoseconds.

[0117] With respect to the high dynamic range electron microscopy methods and systems discussed above, in some cases, the disclosed optical HDR methods (and systems configured to implement them) may include a processor or computer-implemented algorithms and methods for processing acquired data and generating high dynamic range diffraction patterns, images, or video data. For example, in some cases, the processor or computer-implemented algorithms and methods may (i) segment raw camera readout data into subframes containing individual diffraction patterns or images, (ii) correct distortions introduced by the deflection process, and (iii) combine related series / groups of subframes together, each series containing data from two or more subframes having the same measurement parameters except for their exposure times, to generate individual patterns or images with improved dynamic range. In some cases, these processing steps may utilize pre-processed calibration data from the same sample from which the image data was acquired. In some cases, these processing steps may utilize pre-processed calibration data for different samples measured under similar conditions. In some cases, the processor or computer-implemented algorithms and methods may perform additional processing steps such as (i) combining different detection channels (e.g., data sets from different types of detectors) to generate a multi-modal data set; (ii) fine-tuning the sub-frame alignment by identifying consistent features within the sub-frame data itself; (iii) using the fine-tuning of the sub-frame alignment to correct for sample drift during measurement; (iv) filtering and removing noise using prior knowledge of the sample or physical system being measured, the characteristics of the detector, and / or the optical system itself; and / or (v) inferring data from areas of the detector array that are exposed multiple times during a single readout cycle.

[0118] In some cases, the disclosed methods for obtaining high dynamic range optical image data may be implemented in the context of optical microscopy for imaging cells or other microscopic structures. In some cases, the disclosed methods for obtaining high dynamic range optical image data may be implemented in the context of high speed video cameras for imaging moving objects.

[0119] HFR and HDR Systems and System Components: In general, the electron microscopy systems disclosed herein for obtaining high frame rate and / or high dynamic range electron microscopy data (e.g., image data and / or diffraction data) may comprise a transmission electron microscope (TEM) or a scanning transmission electron microscope (STEM), a high speed two-dimensional electron deflector, a deflector control module, a two-dimensional detector or a two-dimensional array of detectors (which in some cases may comprise a CMOS camera or other detector configured with a rolling shutter readout mode), a processor or computer, or any combination thereof. In some cases, the disclosed electron microscopy systems may further comprise software for (i) assembling high frame rate video data from multiple sub-regions or individual detector data sets obtained during a single detector or detector array readout cycle, and / or (ii) processing data obtained from each of the multiple detector sub-regions or individual detectors and compiling a high dynamic range image. In general, an optical system for obtaining high frame rate and / or high dynamic range optical data (e.g., image data and / or diffraction data) may comprise an optical system, a high speed two-dimensional photon deflector, a deflector control module, a two-dimensional detector or two-dimensional array of detectors (which in some cases may comprise a CMOS camera or other detector configured with a rolling shutter readout mode), a processor or computer, or any combination thereof, configured to collect light transmitted, reflected, or scattered by a sample or object and image it onto a two-dimensional detector or detector array. In some cases, the disclosed optical system may further comprise software for (i) assembling high frame rate video data from multiple sub-regions or individual detector data sets obtained during a single detector or detector array readout cycle, and / or (ii) processing data obtained from each of the multiple detector sub-regions or individual detectors and compiling a high dynamic range image.

[0120] 2-D Deflector: Examples of suitable 2-D electronic deflectors for use in the disclosed methods and systems include, but are not limited to, 2-D electrostatic deflectors, 2-D electromagnetic deflectors, or combinations thereof. In some instances of the disclosed systems, two or more 2-D electrostatic and / or electromagnetic deflectors may be combined to implement a deflection system with 3-D deflection capabilities.

[0121] Examples of suitable two-dimensional photon deflectors for use in the disclosed methods and systems include, but are not limited to, a set of rotating mirrors, a set of rotating prisms, a mirror or prism configured to translate along a linear path (e.g., a motorized mirror or prism), or a combination thereof.

[0122] Deflection Control Module: In some cases, the disclosed systems may include a deflector control module configured to (i) control the switching time for switching the deflected electrons or photons from one sub-region or detector to another, (ii) control the dwell time for exposing one or more sub-regions or detectors to the deflected electrons or photons, (iii) synchronize the deflection pattern with a readout signal or readout cycle for the two-dimensional detector or detector array, and / or (iv) synchronize any of the aforementioned parameters with another experimental parameter, e.g., a control signal for positioning the probe beam on the sample.

[0123] Detectors: Examples of suitable detectors for use in the disclosed methods and systems include, but are not limited to, charge-coupled device (CCD) image sensors and cameras, complementary metal-oxide semiconductor (CMOS) image sensors and cameras, electron direct detection (EDD) image sensors and cameras (i.e., CCD or CMOS image sensors modified to include a layer that emits photons when struck by electrons or to directly detect electrons), time-of-flight (ToF) image sensors and cameras, dark-field STEM detectors, Faraday cups, quad photodiodes, annular dark-field detectors, bright-field detectors, general-purpose detectors, or any combination thereof.

[0124] For example, in some cases where an array of detectors is used, different detectors of the array may have different readout modes and / or sensitivities optimized for different electron or photon intensities. In some cases, different detectors may have different levels of protection from radiation damage. In some cases, different detectors may have different objects placed in the path between the sample or object and the individual detectors, for example to selectively occlude bright central spots present in the diffraction pattern data. In some cases, some of the detectors in the array of detectors may be optimized to collect diagnostic, calibration, or alignment data that is qualitatively different from the data obtained by other detectors in the array.

[0125] Processors and Computers: In some cases, the disclosed systems may include one or more processors, computers, or computer systems configured for control and synchronization of the high speed two-dimensional deflector with the readout of a single detector or array of detectors, and for storage, processing, analysis, and display of acquired image data. In some cases, the one or more processors, computers, and computer systems may also be configured for control of other system functions and / or other data acquisition, storage, processing, analysis, or display functions.

[0126] FIG. 11 provides a schematic diagram of a computer system 1101 programmed or otherwise configured to implement methods described elsewhere herein (e.g., methods for obtaining high frame rate and / or high dynamic range electron microscopy data). The computer system 1101 can coordinate various aspects of the disclosed methods and systems, such as, for example, obtaining and processing diffraction pattern data, image data, or any combination thereof. The computer system 1101 may comprise a local computer system, a user's electronic device (e.g., a smartphone, laptop, or desktop computer), or a user's electronic device in communication with a computer system located remotely relative to the electronic device. The computer system 1101 may be a post-classical computer system (e.g., a quantum computing system).

[0127] The computer system 1101 includes a central processing unit (CPU, also referred to herein as a "processor" and a "computer processor") 1105, which may be a single-core or multi-core processor, or multiple processors for parallel processing. The computer system 1101 also includes memory or memory locations 1110 (e.g., random access memory, read-only memory, flash memory), an electronic storage unit 1115 (e.g., a hard disk), a communication interface 1120 (e.g., a network adapter) for communicating with one or more other systems, and peripheral devices 1125, such as cache, other memory, data storage devices, and / or electronic display adapters. The memory 1110, the storage unit 1115, the interface 1120, and the peripheral devices 1125 communicate with the CPU 1105 through a communication bus (solid lines) such as a motherboard. The storage unit 1115 may be a data storage unit (or data repository) for storing data. Computer system 1101 can be operatively coupled to a computer network ("network") 1130 using communication interface 1120. Network 1130 may be the Internet, an intranet and / or an extranet, or an intranet and / or extranet in communication with the Internet. Network 1130, in some cases, is a telecommunications and / or data network. Network 1130 can include one or more computer servers, which may enable distributed computing, such as cloud computing. Network 1130 can, in some cases, implement a peer-to-peer network with computer system 1101, which may enable devices coupled to computer system 1101 to behave as clients or servers.

[0128] The CPU 1105 is configured to execute sequences of machine-readable instructions, which may be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 1110. The instructions may be directed to the CPU 1105, which may then be programmed or otherwise configured to implement the methods of the present disclosure. Examples of operations performed by the CPU 1105 may include fetch, decode, execute, and writeback.

[0129] The CPU 1105 may be part of a circuit, such as an integrated circuit. One or more of the other components of the system 1101 may be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[0130] The storage unit 1115 is configured to store files such as drivers, libraries, and saved programs. The storage unit 1115 may also store user data, such as user preferences and user programs. The computer system 1101 may in some cases include one or more additional data storage units external to the computer system 1101, such as a data storage unit located on a remote server that communicates with the computer system 1101 through an intranet or the Internet.

[0131] Computer system 1101 may communicate with one or more remote computer systems through network 1130. For example, computer system 1101 may communicate with a user's remote computer system (e.g., a cloud server). Examples of remote computer systems include a personal computer (e.g., a desktop PC), a portable personal computer (e.g., a laptop or tablet PC), a smartphone (e.g., an Apple® iPhone®, an Android-enabled device, etc.), or a personal digital assistant. A user may access computer system 1101 via network 1130.

[0132] Software and Algorithms: As discussed above, in some cases, the disclosed systems may further comprise software for (i) assembling high frame rate video data from multiple sub-regions or individual detector data sets acquired during a single detector or detector array readout cycle, and / or (ii) processing data acquired from each of the multiple detector sub-regions or individual detectors and compiling a high dynamic range image.

[0133] In some cases, the methods described herein may be implemented using machine (e.g., computer processor) executable code stored on an electronic storage location of a computer system such as that illustrated in FIG. 11 (e.g., in memory 1110 or electronic storage unit 1115 of computer system 1101, etc.). The machine executable or machine readable code may be provided in the form of software. During use, the code may be executed by the processor 1105. In some cases, the code may be read from the storage unit 1115 and stored on the memory 1110 for quick access by the processor 1105. In some situations, the electronic storage unit 1115 may be omitted and the machine executable instructions are stored on the memory 1110.

[0134] In some cases, the code can be pre-compiled and configured for use with a machine having a processor adapted to execute the code. In some cases, the code may be compiled during run-time. The code can be provided in a programming language that can be selected to allow the code to be executed in a pre-compiled or as-compiled manner.

[0135] Aspects of the methods and systems provided herein, such as the computer system 1101, can be embodied in programming. Various aspects of the technology can be considered as a "product" or "article of manufacture" typically in the form of machine (or processor) executable code and / or associated data carried on or embodied in some type of machine-readable medium. The machine executable code can be stored on an electronic storage unit, such as in memory (e.g., read-only memory, random access memory, flash memory) or on a hard disk. A "storage" type medium can include any or all of the tangible memory of a computer system, computer processor, or equivalent, or its associated modules, such as various semiconductor memory devices, tape drives, disk drives, optical drives, and the like, that can provide non-transitory storage at any time for software programming. All or a portion of the software may be communicated from time to time over the Internet or various other telecommunications networks. Such communication may, for example, enable loading of the software from one computer or processor to another, for example, from a management server or host computer to a computer platform of an application server. Thus, other types of media that may carry software elements include optical, electrical, and electromagnetic waves, such as those used across physical interfaces between local devices, through wired and optical fixed networks, and via various air links. Physical elements that carry such waves, such as wired or wireless links, optical links, or the like, may also be considered software-bearing media. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0136] Thus, a readable medium such as a computer executable code may take many forms, including, but not limited to, a tangible storage medium, a carrier wave medium, or a physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer or equivalent that may be used to implement a database. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include copper wire and optical fibers, including coaxial cables, i.e., the wires that comprise a bus in a computer system. Carrier wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer readable media thus include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM, a DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with a pattern of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave that transports data or instructions, a cable or link that transports such a carrier wave, or any other medium from which a computer may read programming code and / or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[0137] The computer system 1101 may include or be in communication with an electronic display 1135 with a user interface (UI) 1140, for example, to provide an interface for a user to enter instructions, upload data to a computer database, download data from a computer database, etc. Examples of a UI include, but are not limited to, a graphical user interface (GUI) and a web-based user interface.

[0138] In some cases, the methods and systems of the present disclosure may be implemented through the use of one or more algorithms, e.g., algorithms comprising instructions for obtaining and / or processing diffraction pattern data, image data, and the like. The algorithms may be implemented by software responsive to execution by the central processing unit 1105.

[0139] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the present invention. It is understood that various alternatives to the embodiments of the present invention described herein may be employed in any combination in practicing the present invention. The following claims define the scope of the present invention, and it is intended that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

1. 1. A method for obtaining transmission electron microscopy data, the method comprising: a) providing a transmission electron microscope comprising a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample positioned at a sample plane, the two-dimensional deflector being positioned between the sample plane and a two-dimensional detector array; b) modulating a deflector control signal such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of the electrons deflected onto each of the two or more sub-regions; c) synchronizing a modulation of the deflector control signal with a detector array readout signal such that each of the two or more sub-regions captures a data set within each readout cycle of a series of repeating readout cycles of the two-dimensional detector array, the modulation of the deflector control signal being synchronized with a rolling shutter synchronization signal; A method comprising:

2. The method of claim 1 , wherein the two-dimensional deflector is a two-dimensional electrostatic deflector.

3. The method of claim 1 , wherein the two-dimensional deflector is a two-dimensional electromagnetic deflector.

4. A method according to any preceding claim, wherein the two or more sub-regions comprise multiple sub-regions of a single detector comprising a two-dimensional array of individual pixels.

5. The method of any one of claims 1 to 3, wherein the two or more sub-regions comprise a plurality of different detectors in a two-dimensional detector array.

6. A method according to any one of claims 1 to 5, wherein the rolling shutter synchronization is configured to direct the electrons to a sub-region within the series of two or more sub-regions that is not being actively read out.

7. 7. The method of claim 1, wherein the two-dimensional detector array comprises a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, an electron direct detection (EDD) camera, or a time-of-flight (ToF) image sensor, and the modulation of the deflector control signal is synchronized with the rolling shutter synchronization signal for the CCD, CMOS, EDD, or ToF camera such that deflected electrons are directed to a small area that is not being actively read out.

8. 8. The method of claim 7, wherein a duty cycle for data acquisition is at least 90% without incurring time distortion of the data.

9. A method according to any one of claims 7 to 8, wherein the data set captured by each of the two or more sub-regions comprises a frame of image or diffraction data.

10. The method of claim 9, further comprising processing the images or the diffraction data for the two or more small regions to generate video data having a frame rate faster than a readout rate for the CCD, CMOS, or ToF camera.

11. The method of claim 10, wherein a frame rate of the video data set is at least four times faster than a readout rate of the CCD, CMOS, or ToF camera.

12. The method of any one of claims 1 to 11, wherein the switching time for the two-dimensional deflector is less than 10 microseconds.

13. 13. The method of claim 1, wherein the deflector control signal is modulated such that a dwell time for deflecting electrons into each of the two or more sub-regions is different for at least one of the two or more sub-regions.

14. The method of any one of claims 1 to 13, wherein a dwell time for deflecting electrons to the two or more sub-regions ranges from about 1 microsecond to about 100 milliseconds.

15. The method of any one of claims 1 to 14, wherein each of the two or more sub-regions of the two-dimensional detector array comprises an individual detector.

16. A method according to any one of the preceding claims, wherein the data sets captured by the two or more sub-regions are processed to correct for distortions introduced by the deflection step.

17. 1. A system for obtaining transmission electron microscopy data, the system comprising: a) a transmission electron microscope; b) a two-dimensional deflector configured to deflect electrons transmitted or scattered by a sample, said two-dimensional deflector being positioned within said transmission electron microscope between a sample plane and a two-dimensional detector array; c) a deflector control system configured to modulate deflector control signals such that the deflected electrons are sequentially deflected onto a series of two or more sub-regions of the two-dimensional detector array, thereby capturing a data set corresponding to a pattern of electrons deflected onto each of the two or more sub-regions; Equipped with the deflector control signal is synchronized with a detector array readout signal such that each of the two or more two-dimensional sub-regions captures a data set within each readout cycle of a series of repeating readout cycles of the two-dimensional detector array, and modulation of the deflector control signal is synchronized with a rolling shutter synchronization signal.

Images