System and method for extraction of structural data of a sample from scanned data - Patents.com

Abstract

A computer system for determining a structural image of a sample is presented. The computer system receives at least one scanning data set ( IM ) N Each measurement includes data provided by M detection channels associated with an M-segment detector (M≧3). The computer system comprises a data analyzer capable of processing at least one scan data set to compensate for image shifts induced by the off-axis detection channels for sample features in out-of-focus planes, thereby obtaining data indicative of a parallax-corrected scan image of the sample, thereby enabling separation between phase and depth information and extraction of a depth-contrast image of the sample from a single scan data set.

Description

[Technical field]

[0001] The present invention is in the field of the examination of samples by scanning microscopy and relates to systems and methods for the analysis of scanned image data with the aim of extracting structural data about the sample.

[0002] [References] References considered to be relevant as background to the subject matter of this disclosure are listed below. 1.BOSCH,EGT& LAZIC,I.(2019).Analysis of depth-sectioning STEM for thick samples and 3D imaging.Ultramicroscopy 207,112831. 2.MACLAREN,I.,MACGREGOR,TA,ALLEN,CS& KIRKLAND,AI(2020).Detectors-The ongoing revolution in scanning transmission electron microscopy and why this important to material characterization.APL Materials 8,110901. 3.MACLAREN,I.,WANG,L.,MCGROUTHER,D.,CRAVEN,AJ,MCVITIE,S.,SCHIERHOLZ,R.,KOVACS,A.,BARTHEL,J.& DUNIN-BORKOWSKI,RE(2015).On the origin of differential phase contrast at a charged locally and globally charge-compensated domain boundary in a polar-ordered material.Ultramicroscopy 154,57~63. 4.SHIBATA,N.,FINDLAY,S.D.,MATSUMOTO,T.,KOHNO,Y.,SEKI,T.,SANCHEZ-SANTOLINO,G.& IKUHARA,Y.(2017).Direct Visualization of Local Electromagnetic Field Structures by Scanning Transmission Electron Microscopy.Accounts of Chemical Research 50,1502~1512. 5.NORD,M.,WEBSTER,R.W.H.,PATON,K.A.,MCVITIE,S.,MCGROUTHER,D.,MACLAREN,I.& PATERSON,G.W.(2020).Fast Pixelated Detectors in Scanning Transmission Electron Microscopy.Part I:Data Acquisition,Live Processing,and Storage.Microscopy and Microanalysis 26,653~666. 6.OPHUS,C.(2019).Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM):From Scanning Nanodiffraction to Ptychography and Beyond.Microscopy and Microanalysis 25,563~582. 7.ELBAUM,M.(2018).Quantitative Cryo-Scanning Transmission Electron Microscopy of Biological Materials.Advanced Materials 30,1706681. 8. SANG, X., LUPINI, A. R., UNOCIC, R. R., CHI, M., BORISEVICH, A. Y., KALININ, S. V., ENDEVE, E., ARCHIBALD, R. K. & JESSE, S. (2016). Dynamic scan control in STEM: spiral scans. Advanced Structural and Chemical Imaging 2, 6. 9. VELAZCO, A., NORD, M., BECHE, A. & VERBEECK, J. (2020). Evaluation of different rectangular scan strategies for STEM imaging. Ultramicroscopy 215, 113021. 10. GRAAF, S. DE, MOMAND, J., MITTERBAUER, C., LAZAR, S. & KOOI, B. J. (2020). Resolving hydrogen atoms at metal-metal hydride interfaces. Science Advances 6, eaay4312. 11. LAZIC, I. & BOSCH, E. G. T. (2017). Chapter Three - Analytical Review of Direct Stem Imaging Techniques for Thin Samples. IN Advances in Imaging and Electron Physics Vol. 199, Hawkes, P. W. (ed.), pp. 75 - 184. Elsevier. 12. D’ERRICO, J. (2013). Inverse (integrated) gradient (https: / / www.mathworks.com / matlabcentral / fileexchange / 9734-inverse-integrated-gradient), MATLAB Central File Exchange. 13. AMANDINE, V., CEDRIC, M., SERGIO, M. & PATRICIA, D. (2019). An ImageJ tool for simplified post-treatment of TEM phase contrast images (SPCI). Micron 121,90~98. 14.Zhang W,Guo W,Zhang C & Zhao S(2019).An Improved Method for Spot Position Detection of a Laser Tracking and Positioning System Based on a Four-Quadrant Detector.Sensors 19,4722.

[0003] Identification of the above references herein should not be inferred as meaning that they are in any way relevant to the patentability of the subject matter of the present disclosure. [Background technology]

[0004] The examination of a sample / specimen by scanning microscopy, and in particular by scanning transmission electron microscopy (STEM), is based on the instantaneous measurement of the electron flux when the specimen is illuminated by a focused probe.

[0005] Tomographic scans acquired with integrated Center of Mass (iCOM) detectors provide a direct way to map the electro-optical refractive index in a sample [1]. The COM can be defined as the first moment of the projected intensity and is conveniently measured in 4D STEM using pixelated detectors [2]. The refractive index is directly related to the local electric potential, which can reveal atomic number and, in some cases, molecular charge, as long as the intensity modulation in the first order diffraction disk is addressed with sufficient resolution [3].

[0006] STEM with pixelated detectors can be used to acquire the entire scattering or diffraction pattern for each scanned point [5, 6]. This approach gives the greatest versatility since virtual detectors can be defined after acquisition. The main limitation of 4D STEM is speed, which requires a compromise between dynamic range and spatial sampling, assuming that the entire camera must be read for each pixel. A more subtle problem arises for low-dose applications such as cryo-STEM [7], in that the number of measurement elements can exceed the number of illumination electrons for practical scanning rates and real spatial pixel densities.

[0007] US Patent No. 9,312,098 describes a method for investigating a sample in a charged particle microscope. According to this technique, a beam of charged particles is directed from a source through an illuminator to illuminate the sample and is scanned across the surface of the sample, and a detector with multiple detection segments is used to collect the transmitted electrons. The output of the detector is recorded as a function of the scan position, which results in the accumulation of a charged particle image of the sample. Signals from the different segments of the detector are combined to result in a vector output from the detector at each scan position. Such a vector field is mathematically processed by subjecting it to a two-dimensional integration operation, thereby resulting in an integral vector field image. Summary of the Invention

[0008] There is a need in the art for imaging techniques for imaging various samples to allow the extraction of as much information as possible about the structure of the sample, while also reducing the number of imaging sessions and therefore the amount of image data. This is even more essential when using scanning microscopy (electron beam scanning or optical scanning) to examine delicate or radiation-sensitive samples.

[0009] The present invention provides a novel approach for data analysis of image data collected by a scanning transmission microscope. According to the technique of the present invention, structural data (3D data) of a sample located close to the focal plane of a scanning radiation beam can be obtained directly by analysis of single-scan image data detected by a multi-segment detector.

[0010] The term "segmented detector" as used herein refers to an azimuthally segmented detector that represents a corresponding detection channel. Such detectors are commonly used to provide differential phase contrast (DPC) data. In the present application, the segmented detector may be constituted by a pixelated detector (array / matrix of pixels), where multiple groups of pixels centered on a central region represent respective multiple segments of the detector, and the detection channel data (readout of a detector segment) is obtained by summing the signals detected by the pixels of the respective groups. In other words, with the use of an array detector, a detection channel is represented by a virtual detector segment formed by the appropriate sum of the pixel elements of the array detector.

[0011] The present invention provides for the analysis of scanned data acquired in a transmission microscope, which may utilize radiation of charged particles (typically an electron beam) or optical radiation or X-ray radiation. Generally, the scanning mode is implemented via relative displacement between the radiation beam and the sample.

[0012] When considering electron or optical transmission microscopes, the scanning mode is typically implemented by movement of the radiation beam relative to a stationary / fixed sample position. For X-ray transmission microscopes, the scanning mode is typically implemented using a raster movement of the sample (sample stage) while the X-ray beam is fixed.

[0013] The data analysis technique of the present invention provides for extracting a depth-contrast image of a sample from a single scan data set of the sample, i.e., a sequence of N measurements (e.g., measurement points) on the sample acquired by an M-segment detector in a single scanning session. To this end, a single scan data set (I M ) N is processed to extract two parts related to different contributions of the signal being detected: one part corresponds to the true phase contrast (related to the phase delay of the coherent illumination being detected by the off-axis detector segments) and the other part corresponds to the depth contrast, which is expressed in terms of defocus aberration effects.

[0014] The inventors have demonstrated the capabilities of this technique for contrast enhancement through comparison and combination of multiple simultaneously acquired detector signals.

[0015] Thus, according to one broad aspect of the present invention, there is provided a computer system for determining a structural image of a sample, the computer system being configured for data communication with a measurement data provider for receiving raw measurement data provided by a scanning microscope, the raw measurement data being indicative of at least one scan data set acquired in a scanning session, the scan data set (I M ) Ncorresponds to a sequence of N measurements on a sample located proximate to a focal plane of a scanning microscope, each measurement comprising data provided by M detection channels associated with an M-segment detector (M≧3), and the computer system comprises a data analyzer configured and operable to process the at least one scanned data set to compensate for image shifts induced by the off-axis detection channels for sample features in a defocus plane, thereby obtaining data indicative of a parallax-corrected scanned image of the sample, thereby enabling separation between phase and depth information and extraction of a depth-contrast image of the sample from a single scanned data set.

[0016] Preferably, the scan data set includes about 9 or more measurements. The detector may include three or more detection segments.

[0017] The depth contrast image of the sample comprises signed intensities, light and dark, which indicate the degree of defocus.

[0018] The data analyzer analyzes the scanned data set (I M ) N Here, the deshifted DPC ds The method comprises a first data processor adapted to determine data indicative of a parallax-corrected scanned image of the sample, referred to as a parallax-corrected scanned image of the sample, the data processor being preferably further adapted to calculate an integrated differential phase contrast image iDPC1 from the data indicative of the parallax-corrected scanned image of the sample, the iDPC1 providing a more accurate mapping of the electro-optical refractive index of the sample than the known integrated differential phase contrast iDPC due to removal of defocus related artifacts.

[0019] In some embodiments, the data analyzer analyzes the raw scan data set (I M )N to calculate an integrated differential phase contrast image iDPC and to determine from the difference between iDPC and iDPC1 a depth contrast image in which signed intensities, light and dark, represent the degree of defocus indicative of the depth of features in the sample.

[0020] In some embodiments, the depth-labeled phase contrast image is generated from the scan data set (I M ) N The iDPC1 information is formed by displaying the iDPC1 information along with different labelings according to the depth of the feature in the sample, as found from For example, the labeling can include a color scale in a 2D image, or recording the feature in different slices of a 3D image file.

[0021] More specifically, in some embodiments, the first data processor processes the scan data set (I M ) N , and a deshifted scan data set (I1 ds , ...I M ds ) N and then processing the deshifted scan data set to obtain the corresponding differential phase contrast data (DPC ds ), thereby providing data indicative of a parallax-corrected scanned image of the sample. Scanning Data Set (I M ) N In, image I is acquired by each ith detection channel based on N measurements along the scan. i Identifying (x,y) by mapping locations in the image according to the location of the focused scanning beam in the focal plane; Image I i Process at least some of (x,y) to find I iThe image shift (Δx i ,Δy i ) and Determine the opposite shift value and calculate the shift ratio for each image I provided by the i-th detection channel. i (x,y) to obtain a deshifted scan data set (I ds , ...I M ds ) and For all detection channels, the deshifted images (I ds (x,y)...I m ds (x,y)) is processed to obtain differential phase contrast data (DPC ds ), thereby compensating for out-of-focus image shifts induced by the off-axis detection channels and providing data indicative of a parallax-corrected scanned image of the sample. It can be implemented as follows:

[0022] The opposite shift value is i (x,y) N may be determined as the average shift obtained from measurements by each of the M detection channels along the scan, with the shift applied to m with the opposite sign.

[0023] In some embodiments, the data analyzer analyzes a deshifted image (I1 ds (x,y)...I m ds (x,y)) (x,y)) intensity signal sum, thereby providing an extended depth of field.

[0024] The depth of features in a sample is calculated based on the defocus Δz calculation, which is the image shift (Δx i ,Δyi The degree of defocus Δz can be found from processing the image shift (Δx i ,Δy i ) data. The features can be isolated by segmentation techniques in image processing, thus processing the image shift one feature at a time. The defocus Δz provides the depth of the features to provide the depth-labeled phase-contrast image.

[0025] Image shift (Δx i ,Δy i ) for at least some of the pairs of detection channels ij in at least some of the N measurements i and I j Calculate the inter-segment cross-correlation between Δx ij and Δy ij and the image shift (Δx i ,Δy i )

[0026] The invention may be used for the analysis of raw measurement data obtained from a scanning transmission electron microscope (STEM) utilizing an M-segment detector, or said raw measurement data obtained by a scanning confocal microscope utilizing an M-segment detector.

[0027] The computer system may be configured for data communication with a storage device in which raw measurement data is stored and provided, which may be associated with the scanning microscope performing the measurements or may be associated with a stand-alone system.

[0028] According to another broad aspect of the invention, it is a scanning microscope unit comprising a scanning unit for probing a sample surface with focused radiation, detecting radiation transmitted through the sample, and acquiring a scan data set (I) corresponding to a sequence of N measurements on the sample located close to the focal plane of the scanning microscope acquired in a scanning session. M ) N and generating measurement data including: a scanning microscope unit including a detection unit having an M-segment detector (M≧3) located in a Fourier plane relative to a focal plane of a scanning unit, wherein each measurement includes M detection channels associated with the M-segment detector; and a control unit including the computer system described above.

[0029] As explained above, the radiation may be electron beam radiation or light radiation. The scanning unit may perform any known suitable pattern of scanning (such as a raster scan, a circular or spiral-like scan, etc.).

[0030] In addition to the M-segment detector, the detection unit may include a central detector located downstream of the M-segment detector and aligned with a central hole in the M-segment detector.

[0031] It should be noted that detection may further include one or more detectors located downstream and / or upstream and / or surrounding said segmented detector. For example, an upstream detector may collect secondary emission (e.g., optical fluorescence or X-ray emission), a downstream detector may be aligned with a central hole of the segmented detector, and a surrounding detector may be an annular detector operable as a high angle annular dark field (HAADF) detector.

[0032] According to a still further broad aspect of the invention there is provided a method for use in reconstructing a structural image of a sample, the method comprising: At least one scanned data set (I M ) N providing raw measurement data indicative of: processing and analyzing the at least one scan data set to compensate for out-of-focus image shifts induced by off-axis detection channels, thereby obtaining data indicative of a parallax-corrected scan image of the sample, which allows for separation between phase and depth information and extraction of a depth-contrast image of the sample from a single scan data set; Includes.

[0033] In order to better understand the subject matter disclosed herein and to illustrate how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a block diagram showing in outline the configuration and operation of a data analysis system according to the present invention. [Diagram 2] Figure 2A is a flow diagram of a method of the present invention that may be implemented by the system of Figure 1 to obtain a parallax-corrected scanned image of a sample from a single scan of raw measurement data, and how this parallax-corrected image data may be used to extract a depth-contrast image. Figure 2B is a flow diagram of a method of the present invention for determining a parallax-corrected scanned image of multiple objects in a sample. Figure 2C is a flow diagram of a method of the present invention for forming a depth-labeled phase-contrast image. [Diagram 3]3A-3B show images of fibroblasts acquired using a state-of-the-art confocal scanning microscope in simple light collection during transmitted light scanning (FIG. 3A) and (b) the image after application of differential interference contrast (DIC) filtering (FIG. 3B). [Figure 4] 4A-4B are diagrams illustrating the technique of the present invention implemented using the principles of a confocal microscope, where FIG. 4A shows the general principles of the light detection scheme in a confocal scanning microscope and how it can be used in the present invention, and FIG. 4B shows the specifications of the elements in a multi-segment detector (shaded area) and the outer circle that can be determined by the cone of illumination. [Diagram 5] 5A-5D show the results of experiments performed by the inventors comparing the technique of the present invention with a DIC filtering method, where FIG. 5A shows iDPC image data calculated from raw measurement data, FIG. 5B shows a deshifted iDPC (iDPC1) calculated from the original image after applying a deshifting (refocusing) process, and FIG. 5C shows an iDPC2 image corresponding to the difference between the iDPC and iDPC1 of FIG. 5A, revealing a clear image of the cells similar to the DIC filtered image of FIG. 5D. [Figure 6] FIG. 6 shows plots of image shift acquired at several values ​​of defocus that reveal the predicted linear relationship. [Figure 7] Figures 7A-7B are diagrams illustrating a detector suitable for use in a multi-channel detection mode in a STEM that allows analysis of a scan data set according to the present invention: Figure 7A shows a detector with an aperture in the center, inner quadrant (A-D) segments, and outer annular ring (E, F) segments for a separate bright field (BF) detector or EELS spectrometer, and Figure 7B shows a montage of a particular six-channel segment acquired in a real space scan showing the uniformity of the detector response. Figure 7C shows, as a block diagram, an exemplary STEM configuration incorporating the present invention. [Figure 8]FIG. 8 is a diagram illustrating a protocol that may be used in a microscope to compensate for distortions in scanning due to inductive scan delay. [Figure 9] 9A-9D illustrate a pattern of a sequence of scan points, i.e., a pseudo-spiral scan (FIG. 9A), an uncorrected full span of a 10 second scan, i.e., a spiral scan with a perfect circular margin circumscribing a square image (FIG. 9B), and two examples of correction for the central region of a 10 second scan according to the inductive model of FIG. 8 (FIG. 9C) and correction for the central region of a 5 second frame time (FIG. 9D). [Figure 10] FIG. 10 shows a schematic diagram of the analysis of the position of a uniform diffraction disk on a quadrant detector. [Figure 11] FIG. 11 illustrates a histogram of the response of the detector of FIG. 7B in different channels. [Figure 12] 12A-B show a comparison between the diffraction disk position based on four-quadrant analysis (solid line) and the diffraction disk position based on the microscope diffraction shift status readout (dashed line) in two cases: without a specimen and with lens renormalization after each measurement (FIG. 12A), and with an inserted replica grating specimen (FIG. 12B), where the beam is statically deflected using the control panel. [Figure 13] 13A-13B show integrated differential phase contrast data iDPC images based on first and third order terms obtained by analyzing the diffraction disk movement at focus, and iDPC images based on first and third order terms obtained by analyzing the diffraction disk movement at -1.4 μm defocus, respectively. [Figure 14]14A-14D show transmission images of boron nitride nanotubes calculated from the detector quadrant (segments A-D) and ADF (segment E) data of FIG. 7B, where FIG. 14A shows iDPC phase shift by Fourier analysis (iDPCFT), FIG. 14B shows iDPC phase shift by real space integration (iDPCRS), FIG. 14C shows ADF scan (ADF), and FIG. 14D shows the difference image from the sum of segments A-D at two defocus settings (Δf). [Figure 15] 15A-15F show the focus extension of the summed image of the bright field quadrant of a detector segment by compensating for parallax image shift for defocus settings of -1.5 μm, -0.5 μm, and 1.5 μm, where 15A-15C correspond to the readout of a simple annular bright field (ABF) detector by directly summing the signals from the four channels A, B, C, and D, and 15D-15F correspond to the same scans summed after parallax correction by aligning the images by cross-correlation. [Figure 16] 16A-16D show calculations of CTF vs. defocus at three spatial frequencies, showing CTFiS for the particular quadrant detector of FIG. 7B with particular operating parameters (FIG. 16A), CTFiS for a hypothetical quadrant detector without a central hole (FIG. 16B), CTFj2 under similar conditions (FIG. 16C), and the CTF (modulo pre-factor) of the ADF, FIG. 16D. [Figure 17] Figures 17A-17H show the calculated iDPC signal, where Figures 17A-17D show a portion iDPCj (iDPC1) calculated based on a matching quadrant image in which the disparity contribution has been computationally compensated for (by "deshifting"), and Figures 17E-17H show a second portion iDPCj2 (iDPC2) calculated from the remainder of the iDPC signal. [Figure 18] FIG. 18 illustrates the DPC vector field (inversion) for the portion of the data in FIGS. 17A-17H, where the larger magnitude arrows are interpreted as electric fields pointing towards the accumulated negative charge. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Referring to Figure 1, there is shown, as a block diagram, a data analysis system 10 of the present invention for determining a structural image of a sample. The system 10 is generally a computer system comprising, among other things, data input and output utilities 12A, 12B, a memory 12C, and a data processor and analyzer 12D. The system 10 is configured for data communication (via any known suitable type of wire-based or wireless-based technique) with a measurement data provider 14. To this end, the system may include a suitable communication utility 16, which may be of any known suitable type.

[0036] The measurement data provider 14 is generally a storage utility in which the measurement data is stored and may be the storage utility of the scanning microscope itself or an external storage device associated with the scanning transmission microscope (electron microscope, optical microscope or X-ray microscope). Thus, the system 10 may be connected / integral to the scanning microscope for processing the measurement data being collected in real time (in a so-called online mode) or may be a stand-alone computer system that receives the measurement data in an offline mode from a storage device.

[0037] The measurement data that can be processed by the system 10 to provide a structural image of the sample is raw measurement data obtained from a segmented detector that collects illumination transmitted through the sample during a single scanning session in a scanning microscope, which can be an electron beam scanning session or an optical scanning session.

[0038] Thus, raw measurement data refers to a scan data set (I ) that includes a sequence of N measurements (e.g., measurement points) (N≧9) on a sample collected in a single scan and acquired by an M segment detector (M≧3). M ) NThe scanned data set is processed and analyzed by the data processor and analyzer utility 12D. The data processor and analyzer utility 12D processes and analyzes the scanned data set (I M ) N The data representing the parallax-corrected scanned image data (structural image) of the sample is processed to obtain the so-called deshifted differential phase contract data DPC. ds The system includes at least a first processor 18 configured and operable to provide:

[0039] Preferably, this first processor 18 also calculates the corresponding integrated differential phase contrast data iDPC1. This allows for separation between phase and depth information and direct extraction of a depth contrast image of the sample, where the signed intensities, light and dark, represent the degree of defocus. To this end, the data processor and analyzer preferably also includes a second data processor which utilizes data indicative of the integrated differential phase contrast data iDPC (calculated from the scan data set by either the first processor or the second processor) and determines a difference iDPC2 between iDPC and iDPC1, which represents a depth contrast image of the sample, where the signed intensities, light and dark, represent the degree of defocus, which is indicative of the depth of features in the sample.

[0040] The data processor and analyzer 12D may therefore be configured and operable to determine two portions in the scan data set relating to different contributions of the signal being detected: one portion relating to a parallax component induced by the phase delay of the coherent illumination being detected by the off-axis detector segments and representing the true phase contrast (determined by the structure / composition of the sample), and the other portion relating to a parallax component dependent on defocus aberration effects.

[0041] More specifically, as shown in FIG. 1 and further illustrated in flowchart 100 of FIG. 2A, the processor and analyzer 12D can operate to compensate for image shifts induced by off-axis detection channels for sample features in out-of-focus planes in a scan data set, thereby separating between phase contrast and depth contrast in the image data and directly extracting a depth-contrast image of the sample from a single scan data set.

[0042] Such a process may be implemented as follows.

[0043] As indicated above, a scan data set (I ) includes a sequence of N measurements / points for a sample, each acquired by M detection channels of an M-segment detector (M≧3). M ) N is provided / received (step 102). Each i-th detector segment generates an image I based on N measurements along the scan. i (x,y) is provided by mapping locations in the image according to the location of the focused beam exactly in the focal plane.

[0044] The scan data sets are processed (step 104) to determine a parallax-corrected scan image (stage 108). For this purpose, a data set I i For at least some of the N points in (x,y), the image shift (Δx i ,Δy i ) is determined. This is the (x,y) coordinate of the focused beam at the focal plane. foc location and (x,y) of the beam on the sample in the defocus plane, i.e., at a z location outside the focal plane. defoc location (eg, a common shift for all N points acquired by the i-th detection channel).

[0045] In other words, I iThe image shift between (x,y) and a hypothetical image acquired from a central region of the detector at the optical axis defined by the illumination path of the microscope is found (step 108). Determining the shift can be implemented, for example, as follows:

[0046] All possible vectors of relative image shifts are calculated for at least some of the pairs of detection channels ij in at least some of the N measurements, i.e., for the images I i (x,y) and image I j An inter-segment cross-correlation is performed between at least some of the (x,y) and the relative shift Δx ij and Δy ij is calculated, and the image shift value (Δx i ,Δy i ) is determined from the best fit terms (step 110), which is described in more detail further below.

[0047] The opposite shift value is then applied to each normalized signal I i (x,y) to obtain the corresponding deshifted image I for each i-th detector segment / detection channel. i ds (x,y) (step 112). Each set of N samples / measurements is now permuted / shifted differently, and the resulting deshifted scan data set (I1 ds , ...I M ds ) is used to extract the deshifted differential contrast data DPC ds (Step 114). This deshifted differential phase contrast (DPC ds ) shows a parallax-corrected scanned image of the sample.

[0048] Preferably, this data DPC dsis used therefrom to calculate a corresponding integrated differential phase contrast (iDPC1) (step 116), which provides a parallax-corrected scanned image (detected signal related to the radiation (light or electron beam) passage through the structure / composition of the sample) that relates to the true phase of the sample, and exhibits an extended depth of field (as described in further more detail below).

[0049] Additionally, and preferably in some embodiments, the scanned data set (I M ) N are also processed to determine integrated differential phase contrast iDPC data (step 106), which results in the original scan data set (I1, ...I M ) and the integrated differential phase contrast data (iDPC) of the deshifted scan data set (I1 ds , ...I M ds ) it is then possible to determine the difference iDPC2 between the integrated differential phase contrast data iDPC1 (step 118), thereby enabling direct extraction of the defocus-dependent disparity component of the image, providing a simple and highly interpretable depth contrast.

[0050] The data indicative of the disparity-corrected scanned image may be so-called global disparity-corrected image data of multiple features / objects within a range of interest R. This technique is illustrated by a flow chart 120 in FIG. 2B. For ease of understanding, similar steps / elements in the various examples of the invention described herein are identified by the same reference numbers. As shown in the figure, raw single scan data (I M ) N is provided (step 102), and for every feature / range of interest R, the following process is performed: A set S(R) of pixels in the image that belong to the feature / range of interest R is determined (step 122). Then, the i-th detector segment image I is extracted by removing pixels outside of said set S(R). i (x,y) to image I i,R(x,y) are generated, step 124. Image shifts are determined for Ii,R(x,y), similar to step 110 described above, and these shifts determine the deshifted images and corresponding deshifted DPCs, similar to steps 112 and 114 described above. ds is then used to calculate the DPC ds At point S(R) according to (R), the global DPC ds The value of DPC is determined (step 126). As in the example described above, ds is preferably used to calculate a corresponding integrated differential phase contrast (iDPC1) therefrom, step 116.

[0051] As indicated above, the technique of the present invention also includes a method for detecting the presence of a scanned data set (I M ) N The present invention provides for obtaining a depth-labeled phase contrast image by displaying iDPC1 information along with a differential labeling of features in a sample according to depth found from. For example, the labeling may include recording a color scale in a 2D image, or features in different slices of a 3D image file. In this regard, reference is made to FIG. 2C, which illustrates a flow chart 130 of the present technique.

[0052] As shown, the raw single scan data (I M ) N is provided (step 102), and for every feature / range of interest R, the following process is performed: a set S(R) of pixels in the image that belong to the feature / range of interest R is determined (step 122), and the i-th detector segment image I is extracted by removing pixels outside said set S(R). i (x,y) to image I i, R(x,y) are generated, step 124. The image shift found in step 110 described above is then used to calculate the defocus z(R), for example using model-based processing (step 136). Then, according to the degree of defocus Z(R), a color or any other suitable labeling mark C(R) can be selected, step 138, and pixel S(R) is painted according to the intensity of iDPC1 (determined in step 116 described above) in the selected color (label) C(R), step 140.

[0053] The data analysis techniques described above can be used with electron beam scanning microscopes or optical microscopes or X-ray microscopes by using any known suitable microscope configuration, with the only modification that multiple detection channels are used to detect illumination transmitted through the sample (either by replacing a pixelated detector with a segmented detector, as described above, or by appropriate interpretation of the pixelated detector readout).

[0054] Considering optical microscopy, the principles of the present invention can be implemented in a modified confocal scanning microscope by using a segmented detector. Biological samples, such as cells without staining, are mostly transparent to light. Certain features or molecules in can be fluorescent, whether naturally or by labeling with fluorescent proteins or chemical dyes. Generally, confocal microscopes are designed to achieve contrast in such fluorescent samples and to scan the inner layers with a resolution on the order of a few micrometers in thickness. The scanning probe is focused by the microscope objective, which means that the illumination cone is defined by the numerical aperture of that objective. The scanned image is acquired by a detector that collects the light returned along the illumination path. An aperture is used to improve the depth selectivity by blocking out-of-focus light, and the aperture is located in the plane of confocal with the specimen focus. Fluorescence is selected from the returned light using a spectral filter. Reflected light can also be used in the case of reflective specimens, such as in metallurgy or mineralogy, but is not sufficient for biological specimens. Often, provisions are made to collect transmitted light as well, but the resulting transmission image does not benefit from a depth-selective aperture. To display the pattern of fluorophores within the context of the cell, both fluorescence (i.e., filtered reflection) and transmitted light scans are required. Because typical specimens are transparent, very little contrast is obtained by simple transmitted intensity. In the current state of the art, contrast in a transmission scan can be achieved with a polarizer and a Nomarski-modified Wollaston prism in the light path, which implements the differential interference contrast (DIC) technique.

[0055] 3A-B illustrate images that can be acquired using a state-of-the-art confocal scanning microscope, where FIG. 3A shows an image resulting from simple light collection and FIG. 3B shows an image after DIC filtration. In this particular example, a transmission scan of fibroblast tissue culture cells on an Olympus Fluoview scanner with a BX50WI microscope, where a DIC component is subtracted or inserted, clearly shows the benefit of DIC filtration. However, image position and shading depend on the polarizer and prism settings, and thus the positioning of the fluorophore distribution in the cellular environment is compromised. Moreover, the complexities of adjusting some optical components clash with other optical operations.

[0056] To implement the techniques of the present invention, a confocal scanning microscope is modified by using a segmented detector, e.g., a quadrant detector, placed in the diffraction (Fourier) plane of the scanning probe of the confocal microscope.

[0057] In this connection, reference is made to Figures 4A and 4B. Figure 4A shows the general principle of the light detection scheme in a confocal scanning microscope 200 and how it can be used in the present invention. As shown in the figure, the illumination cone is structured such that it is focused to probe the focal plane, and a quadrant detector 202 with segments A, B, C and D is located in the diffraction plane. The use of a segmented detector allows the measurement of the differential phase contrast (DPC) and its derived integrated differential phase contrast (iDPC) as follows: TIFF2025507253000002.tif36170

[0058] DPC and iDPC are the normalized currents I received by each i-th detector segment simultaneously from segments A to D. i where the current is proportional to the collected illumination falling on the area of ​​the segment.

[0059] The detected signal is expressed as a complete scanned image I by the i-th detector segment. i (x,y), so that index (x,y) corresponds to the location of the scanning probe on the sample (i.e., the focal point of illumination). The x and y components of the DPC are with respect to the diffraction plane (Fourier plane) where the detector is located, so in the special case where the beam is bent (similar to the bending of the scattering trajectory of electrons in a magnetic field), the corresponding vector should be transformed into the (x,y) framework of the scan (in some cases, the two frameworks are the same). In either case, the DPC is i ...I M It corresponds to the element-wise mathematical operations between

[0060] As mentioned above, and as will be explained in more detail below, the measurement data analysis technique of the present invention utilizes the separation of DPC and iDPC into two parts that relate to different contributions in the signal being detected. One part is the true phase contrast, which relates to the delay in the phase of the coherent illumination by the sample, and the other part is called depth contrast, which involves the effects of defocus aberrations. According to this technique, the true phase contrast (iDPC1 in FIG. 2) is the signal I detected by each ith detector segment along the scan. i From the (x,y) “deshifted” scanned image, i.e., the image shift (Δx i ,Δy i ) and transforming each image by the opposite shift. In effect, this is i This means that (x,y) is synchronized with the true point of (x,y) on the sample.

[0061] The inventors have found that computationally aligned or "deshipped" iDPC1 is robust to lens aberrations and remains nearly the same when images are scanned over a range of lens defocii as large as the canonical depth of field, and even slightly larger.

[0062] The second contribution of iDPC is iDPC-iDPC1=iDPC2 It is determined as iDPC2 image intensity is sensitive to the depth of sample features and therefore it reveals an ordering of features in height based on differences in brightness of similar features.

[0063] There is some freedom in determining the 0 of iDPC2, which represents the background. For presentation of iDPC2 as a depth contrast, iDPC1 is calculated from image shifts of more than about 1 pixel, otherwise that detail cannot be observed. The region of interest (ROI) in iDPC1 calculation for this purpose is preferably at the margins of the image, where contrast may disappear. If the image shift is less than 1 pixel, the value of 1 pixel may be used to de-shift the image.

[0064] Thus, the techniques of the present invention advantageously provide for obtaining 3D information of a sample (i.e. structural data of the sample) in a single scan.

[0065] In the following, the general relationship between defocus and image shift is described using an M-segment detector (M≧3) located in the diffraction plane (Fourier plane).

[0066] By selecting a portion of the image centered on a chosen feature, the amount of defocus (lens correction) needed to scan that feature in focus can be determined. It is also possible to select the entire scanned image and determine the best defocus needed to bring the entire scan to the best average focus, from which relative height or depth is seen as relatively bright or dark intensity.

[0067] FIG. 4B shows the specifications of the elements in the detector (shaded area) and the outer circle that can be determined by the cone of illumination, where τ1 denotes the half angle of the cone. The angles τ0 and τ1 are the arctangent of the radius of the collected illumination divided by the imaging length in an electron microscope, and in an optical system τ1 is simply the numerical aperture. The inner angle τ0=0 unless there is a hole in the center of the detector. The outer angle is the smaller between the beam illumination cone angle and the detector maximum collection angle.

[0068] The azimuth angle of the center of each i-th detector segment with respect to a given x-axis in the Cartesian coordinate system is given by A scanned image is acquired from each detector segment, which, due to its designated location, is compared to a scanned image that would be acquired from a central element / segment in the detector that is on the optical axis defined by the illumination path (Δx i ,Δy i ) is shifted.

[0069] In general, lenses can introduce aberrations. The inventors have determined the relationship between the image shift, the defocus Δz, and the dominant aberrations of the i-th detector segment according to a set of linear equations as follows: TIFF2025507253000004.tif17170, where TIFF2025507253000005.tif12170TIFF2025507253000006.tif12170Here, (A 1x +iA 1y ) is double astigmatism and C3 is spherical aberration.

[0070] For each i-th detector segment, there is a different equation, so the defocus Δz and the aberration constants can be extracted by solving for them when the image shift is known. Often it is possible to ignore all aberrations except for the defocus, and just use the quadrants in a Cartesian coordinate system as A 4-quadrant detector aligned as TIFF2025507253000007.tif12170, i=1..4 may be used, in which case the calculation simplifies to: TIFF2025507253000008.tif13170, TIFF2025507253000009.tif14170, TIFF2025507253000010.tif15170, TIFF2025507253000011.tif12170 In practice, the defocus Δz is obtained by averaging all shift results according to: TIFF2025507253000012.tif11170

[0071] The image shift is usually obtained by finding the location of the highest peak in the correlation map between the two images via digital analysis. However, the reference "non-shifted" scan (from the central detector element) can be omitted and the required image shift is more reliably extracted from all possible pairs between the M shifted images. The procedure is proved as follows for M=4.

[0072] The unknown image shift for a virtual "unshifted" image is given by the shift vectors r1, r2, r3, r4 (r i =(Δx i ,Δy i )) where the relative shift of image q with respect to image p is r pq It is shown that r pq =r q -r p It is related to the required vector as r pq =-r qp , it is sufficient to measure the cross-correlation of six image pairs and the following equation is used: 3r1-r2-r3-r4=r 21 +r 31 +r 41 3r2-r1-r3-r4=r 12 +r 32+r 42 3r3-r1-r2-r4=r 13 +r 23 +r 43 3r4-r1-r2-r3=r 14 +r 24 +r 34

[0073] Known (Δx pq ,Δy pq ), then the required r can be calculated using the Moore-Penrose pseudoinverse solution known in the art. i =(Δx i ,Δy i ) can be solved.

[0074] r pq Note that can be found to sub-pixel resolution as follows: A 16x16 point centered on the highest peak in the cross-correlation map is first isolated and then interpolated (e.g., using the Matlab code fftInterpolate written by Matthias Christian Schabel) to a 512x512 point. The location of the peak is then found to within 1 / 32 pixel resolution.

[0075] Because Δz is determined by a linear relationship to a measurable characteristic (image shift), the performance of existing algorithms that determine defocus based on image blur can be exceeded by the techniques of the present invention because the blur characteristic depends quadratically on the defocus and is therefore less accurate for determining small defocus.

[0076] The technique of the invention is illustrated with a quadrant detector replacing the bundle of optical fibers at the back of the BX50WI microscope, which normally carries the transmitted light to a photomultiplier detector. The quadrant photodiode detector (UDT, SPDT 40) is rectangular and has a total size of 7 mm. The active surface of the detector is placed at the location where the diffraction plane of the condenser is projected, where normally the edge of the fiber bundle is otherwise positioned on the BX50WI. The precise location of the diffraction plane is found using a Bertrand lens replacing one of the eyepieces, where the emitting fiber bundle should appear in focus. The position of the collimator is then aligned by observing the edge of the field iris through the normal eyepiece. By partially contracting the field iris and focusing on its edge, it is ensured that the diffraction pattern of the sample is projected onto the selected plane of the detector.

[0077] For demonstration, the four wires and the ground pin of the quadrant detector are connected to a four-channel oscilloscope (Siglent SDS 1104x). The labeling of the channels should follow the order shown in Figure 4A so that the DPC calculations can be applied. Acquisition settings are 250KS / s, 2s / div for a scan time of 28s. Sensitivity is either 10mV / div for AC coupling or 100mV / div for DC coupling, the former being required for sufficient resolution of details in transparent cells. The scan is controlled from Fluoview software and set to 35.9s and 2048x2048 pixels, using zoom 2 and a X20 NA=0.7 objective, thus giving a pixel size of 0.17 microns. The data is stored in a CSV file and analyzed by custom-written MATLAB code to yield images.

[0078] Cells are fixed on a slide and a green laser probe is focused on them with the normal objective focus. The collimator is then realigned so that the fins of the field stop observed through the eyepiece are in focus. This step ensures that the quadrant detector is in the diffraction plane. The signal is acquired by laser scanning and using a custom written Matlab code "confocal_measure.m", the image shift is calculated either over the entire scan area or centered on the feature of interest. Based on that image shift, the defocus is determined according to: TIFF2025507253000013.tif10170

[0079] One of the scans at an arbitrary sample position "z=0" provided the following image shifts: i=1,(8.0859,-7.1797) i=2,(7.0078,6.1797) i=3,(-7.1953,7.6328) i=4,(-7.8984,-6.6328) Image shift average = 7.2 pixels

[0080] Reference is made to Figures 5A-5D which show the results of experiments performed by the inventors comparing the technique of the present invention with a DIC filtering method.

[0081] iDPC image data (FIG. 5A) can be calculated from the raw measurement data as described above with reference to FIG. 4A. Deshifted iDPCs (iDPC1 in FIGS. 1 and 2) are calculated from the images (using custom-written Matlab code confocalb.m) after applying the following translation vector, which undoes the mean shift: i=1,(-7.2,7.2) i=2,(-7.2,-7.2) i=3,(7.2,-7.2) i=4,(7.2,7.2)

[0082] The deshifted iDPC in Fig. 5B appears to deshift one of the layers that is out of focus, but does something wrong to another layer that was originally in the correct position. However, the subtraction iDPC2 = iDPC - iDPC1 shown in Fig. 5C reveals a clear image of the cells, which resembles the DIC filtered image described above (Fig. 5D). It should also be understood that the techniques of the present invention can be used without introducing new optical components into the microscope system.

[0083] Figure 6 shows a plot of the image shift acquired at several defocus points (the sample is moved relative to the objective). The plot is found to be linear, which confirms the modeled theoretical relationship between defocus and average image shift. From the slope of the plot, it can be seen that the numerical aperture should be NA=(3π*0.17 / 4) / 0.809=0.5, which is slightly smaller than the NA=0.7 of the objective.

[0084] It should be noted that in the measurements, several technical improvements can be used to affect both the convenience and the quality of the data, such as the use of appropriate amplifiers for the photodiodes, the use of dedicated signal acquisition devices synchronized to the scan, and the use of an optical switch between conventional transillumination and the use of a segmented detector in scanning mode.

[0085] As noted above, the data analysis techniques of the present invention may be used with a scanning transmission electron microscope (STEM) of any known suitable microscope configuration that utilizes a segmented detector for detecting the response of a sample to electron beam illumination.

[0086] In general, STEM is based on the instantaneous measurement of electron flux when a specimen is illuminated by a focused probe. Various detectors that subtend different solid angles of scattering can be employed, either alone or simultaneously. These can be configured such that their signals are dominated by distinct scattering processes. Thus, multi-segment detectors can report on coherent and incoherent scattering, or distinguish light from heavy elements, based on comparative signal strength [1].

[0087] Simultaneous acquisition of annular dark field (ADF) for differential phase contrast (DPC) and integrating area detectors, such as ring geometry, for off-axis elements are required. Calculation methods such as integrated DPC (iDPC) provide a powerful extension in that image contrast does not have to be a simple representation of the analog signal from the detector.

[0088] In wide-field TEM, the concept of an image as a 2D array of square pixels is inherent to the camera architecture. By convention, STEM images are generated by scanning the probe in a raster pattern and synchronizing a detection window to define pixels in rows and columns, usually with an aspect ratio of 1. This 2D array is kept as a convenience for presentation and storage in standard file formats, but in effect the raw signal is a temporal trace. Separating the measurement sampling from the image pixels leads to considerable freedom in the generation of non-conventional scan patterns and in measures for minimizing damage to radiation-sensitive specimens. Non-conventional scanning has recently been explored in the context of compressed detection acquisition, which could potentially provide improvements in dose efficiency and scan time.

[0089] The present inventors have also developed a scanning and data acquisition control system for STEM that utilizes multi-channel detection based on the use of a multi-segment detector. This technique provides simultaneous multi-channel (e.g., 8-channel) acquisition and arbitrary waveform scanning capabilities (with virtually no practical limitations on the number of channels defined by the detector segments).

[0090] In this regard, reference is made to Figures 7A-7C, where Figures 7A and 7B illustrate multi-segment detectors 300A and 300B, respectively, and Figure 7C illustrates, as a block diagram, a STEM 400 equipped with a scanning and data acquisition control system 402.

[0091] As shown in Fig. 7A, detector 300A includes the following segments providing corresponding detection channels: four-quadrant detector segments A, B, C, D, outer annular ring segments E, F, and an additional detector BF, which is a bright field detector or EEL spectrometer located in the central region of disk 302. In the example of Fig. 7B, detector 300B is generally configured similarly to detector 300A, i.e., includes detector segments A-F, but does not have a central detector, leaving a central hole / aperture. This allows for the placement of additional detector BF downstream of disk 302, while being vertically aligned with the central aperture of the disk.

[0092] The detector of Figure 7B can be implemented similarly to a solid-state segmented detector, commercially available from Opal, El-Mul Technologies, Israel. The figure also shows a montage of six such detector channels acquired in a real space scan, providing uniformity of detector response.

[0093] As shown diagrammatically in FIG. 7C, the STEM 400 is of a commonly known configuration (e.g., FEI Tecnai T20-F), i.e. includes an electron beam column EM configured to implement a scan mode along the x-axis and y-axis and suitably controlled by user input via a user interface and common serial software. However, in this system, the detection unit is configured to implement 8-channel detection and the internal scan generator is modified to implement the scan mode control according to the invention and, possibly, the scan data set analysis described above. Alternatively (or in addition), as shown in the figure with dashed lines, the scan data set analyzer 10 can be part of an external system in data communication with the STEM. The detection unit includes the six-segment detector 300B exemplified above, a bright field detector BF located downstream of the detector 300B and aligned with its central hole, and a high angle annular dark field detector HAADF fitted upstream of or around the detector 300B, providing eight detection channels in total. As also shown in the figure, a scan controller operates to control an arbitrary waveform generator (AWG) to perform x-axis and y-axis scanning and readout circuitry including analog-to-digital converters (ADCs) to convert the electrical output of the eight detector channels. The acquisition hardware is built from off-the-shelf components. Software is organized to make the system look like a camera to the popular microscope control platform SerialEM, which facilitates integration into high performance protocols such as automatic acquisition or tomography.

[0094] The use of the above described data analysis techniques of the present invention for the analysis of signals acquired by multiple detection channels associated with multiple segments of a detector proves that the additional terms in the contrast transfer function for iDPC-STEM or for iDPC optical microscopy reflect the material contrast directly related to phase and the parallax component dependent on defocus. The latter provides a simple and highly interpretable depth contrast. Compensation of the parallax shift provides an extended depth of field and suppresses contrast inversion in the phase image.

[0095] Using non-planar networks of boron nitride nanotubes, various imaging modalities are demonstrated below.

[0096] In general, four design features that should be considered for a flexible scanning system include scan pattern, synchronized multi-channel acquisition, software integration, and data structure.

[0097] Regarding the scanning pattern, the following should be noted: Raster scanning is the most natural way to fill a Cartesian plane, with fast scans in one direction and slow steps in the other. Raster scanning also conveniently corresponds to the storage of data arrays in a computer, row and column by row. In general, one should simply synchronize the sampling to produce a 2D image, similar to the camera readout. However, raster scanning is not a natural way to steer an electron beam. Both the magnetic deflectors and the electronic amplifiers that drive them have a minimum response time, which means that the actual beam location lags behind the control signals that determine the recorded pixel positions. At the end of each line, the beam should stop and reverse direction quickly. This causes very strong scanning distortions near the edges of the frame, where severe damage often accumulates. The displayed field of view is usually cropped to a smaller area than where the scan is properly linear. Significant fragments may have to be discarded, and the displayed area may also shift horizontally depending on the scanning speed.

[0098] A more natural way to scan would be to minimize the change in probe acceleration. For example, a circular scan is perfectly smooth, with sinusoidal and cosine driven orthogonal directions. The probe lag is then equivalent to a phase delay with respect to both. By slowly reducing the amplitude, a shrinking spiral is obtained, or a set of concentric circles is obtained. Various spiral scanning methods have been explored before [8]. Alternatively, a plane can be covered by slowly sweeping a large circle along a line. The Hilbert pattern is another attractive scanning option to reduce distortion by shortening the flyback path [9].

[0099] Maximum flexibility can be achieved by pre-preparing an array of scan coordinates. A Cartesian pixel grid can be reconstructed by interpolation between sampled points that takes into account the phase delay.

[0100] Regarding synchronous multi-channel acquisition, the following should be noted: To improve quantitative comparison between measurements on different channels, the acquisition should be truly simultaneous. Many digitizers time-multiplex the measurements to use a single analog-to-digital converter. This approach can cause aliasing artifacts when sampling close to the clock speed, and moreover, it is difficult to increase the number of channels without proportionally slowing down the acquisition. Simultaneous acquisition is therefore desirable. It is also desirable to sample at a frequency significantly higher than the time response of the detector amplifier. This is useful for noise reduction by averaging and for optimal interpolation of non-Cartesian scans.

[0101] Also, data collection sessions for automated operation, such as through-focus series, tomography and recording for single particle analysis, require a level of meta-control beyond that of single image recording, so appropriate software integration may be required. A commonly used SerialEM package may be implemented in the scan generator to leverage its capabilities for navigation, acquisition and microscope control. It will be appreciated that integration with other software is also possible.

[0102] Requirements for a file format to store multi-channel images with flexible scanning patterns may include efficient data compression, flexibility and ease of handling in field definition, and aggregation of multiple scans in tomography. Metadata may be stored in the same file. A current mature technology that realizes these requirements is the MAT file by Mathworks, which can be directly loaded into MATLAB or processed by available open source libraries based on published formats, such as the popular MRC format

[20] .

[0103] The hardware used in the experiments carried out by the inventors to prove the technique of the present invention is based on a computer card from Spectrum Instruments GmbH (Germany), namely a 2-channel arbitrary waveform generator (AWG) M2p.6541-x4, an 8-channel 16-bit analog-to-digital converter (ADC) M2p.5923-x4, and a STAR-HUB to synchronize the cards. The AWG outputs are tied to the "line" and "frame" external scan inputs for the STEM (hereafter scanX and scanY). 75 ohm external terminators are added at the high impedance microscope inputs. The scanning process starts with the uploading of the pattern vectors for the microscope's scanX and scanY inputs to the on-board memory of the AWG card. The sampling rate, duration, and amplitude are set to determine the field of view and resolution, including margins that will not be part of the image. Synchronization generation and acquisition are then handled by the STAR-HUB. Finally, the acquired recordings are downloaded to a storage utility. The internal storage is sufficient for an 8-channel scan of 2048x2048 pixels with oversampling and scan margin, but in principle a first-in-first-out (FIFO) mode could utilize computer RAM to expand the size.

[0104] Scanning may require distortion compensation due to inductive scan delay. The expected lumped circuit for the beam deflector is a resistor and an inductor in series (as shown diagrammatically in FIG. 8). The location of the beam is determined by the magnetic field and therefore by the current passing through the inductor L in the scan coil. The commanded location of the beam is determined by the voltage generated by the arbitrary waveform generator (AWG) channel divided by the constant resistance R of the resistor. The delay of the current after the voltage has a characteristic time τ=L / R, and therefore the actual position, lags the command signal. A list of digital-to-analog conversion (DAC) vectors holds target points (scanXi, scanYi) for feeding the scan coil, which correspond to the location of the beam in static conditions. The sample locations (xi, yi) in the reconstructed image correspond to the location of the beam in dynamic conditions, determined by the current in the magnetic coil, which lags the DAC signal.

[0105] The delay can be compensated for by adjusting the position keys used to reconstruct the image. The relationship between the commanded location X (or Y) and the actual location Xcorr (or Ycorr) is given by the first-order differential equation Determined by TIFF2025507253000014.tif12170. In discrete form, the equation becomes the corrected sequence at scan position n>1. TIFF2025507253000015.tif11170

[0106] The delay constant fitted to the microscope (FEI, Tecnai T20-F) was found to be about 200 μs. Comparing different scan amplitudes and times, the inventors identified a second-order correction as the dependence of τ on the scan speed. By analyzing the images of the replica grating, τ x and τ yIt has been found that should be adjusted independently to remove kinks in vertical and horizontal lines, respectively. In summary, τ x =A1*[1-B1*(scanx_amplitude / fullXsize) / Δt] τ y =A2*[1-B2*(scany_amplitude / fullYsize) / Δt] In tests performed by the inventors, the fitted values ​​were A1 = 220 μs, A2 = 265 μs, B1 = B2 = 0.1 μs / mV, however, it should be understood that different instruments may require slightly different corrections.

[0107] The scan may require resampling to a 2D image, which may be implemented as follows: The raw data series is converted to a 2D image in Cartesian coordinates for processing and presentation. This involves first correcting for time delay as explained above. The non-raster scan is interpolated to a Cartesian grid of image pixels. With oversampling, the raw data may be denser than the target array. In the acquisition, the oversampling factor may be set by default, e.g., to 10, and should be reduced if the recorded size or sampling rate exceeds the hardware limit of 512GS / 20MS / s. The interpolation is based on the average of nearby (<1 pixel) sampled values ​​around the filled pixel, weighted according to the distance to the sampled location, i.e. The file is TIFF2025507253000016.tif6170. Here, a bilinear weighting factor is used according to the distance (in pixels) between the exact beam position and the pixel center, w i =(1-|xx i |)(1-|yy i |) Bicubic weighting or other weightings may also be implemented.

[0108] Note that in contrast to compressed sensing acquisition, the technique used in the present invention oversamples the equivalent pixel grid. This is obtained at no cost in exposure since there is no fast blanker, the beam is sweeping across the sample anyway, and the analog-to-digital conversion (ADC) bandwidth is larger than the bandwidth of the detector response. Oversampling provides a measure of redundancy and noise reduction compared to instantaneous sampling directly coupled to the pixel lattice.

[0109] To validate the scans used in the experiments, a standard replica grating (S106 with 2160 lines / mm, Agar Scientific) was used to develop several scan patterns.

[0110] It should be noted that the techniques of the present invention are not limited to the type of scan used in the measurement (e.g., raster scan, circular scan, spiral scan, etc.) In general, in measurement data analysis, what needs to be known (and appropriately provided) is data indicative of the arrangement of detector segments to enable appropriate identification and processing of data pieces associated with different detection channels.

[0111] FIG. 9A illustrates the pattern of the sequence of scanning points, i.e., a pseudo-spiral scan consisting of a series of concentric circles with radii decreasing by one pixel starting from a circle circumscribing the requested square image. The fraction of the scanned area that is kept is then 2 / π. In many applications, such as imaging of abundant particles, a square image may not be required, and therefore the entire scan area may be used. The number of sampled points (x,y) is equal to the circumference of the circle times the oversampling factor samples_per_pixel provided for noise reduction as described above. The speed of the beam progression is constant and smooth, except for jumps of one pixel between the circles at certain angles.

[0112] Figure 9B illustrates the uncorrected full span of a 10 second scan, i.e. a spiral scan with perfectly circular margins circumscribing a square image. Uncorrected artifacts include a twist in the center of the scan and a displacement of lines that should appear straight. Figures 9C and 9D show two examples of correction for the central region of a 10 second scan (Figure 9C) and a 5 second frame time (Figure 9D) according to the inductive model described above with reference to Figure 8. Residual distortion remains in the faster scans, which can be corrected by higher order time derivatives. At the end of the scan, the beam is deflected to one of the outer corners of the image frame.

[0113] Although the scan data sets from each detection channel associated with a detector segment (e.g., of the detector illustrated in FIG. 7A or FIG. 7B) can be stored separately resulting in multiple images, the signals are not independent and the power of the segmented detector occurs in combination between those channels.

[0114] Reference is made to FIG. 10, which shows a schematic analysis of the position of a uniform diffraction disk on a quadrant detector. The sum of the four quadrant detector signals I1, I2, I3 and I4 relates the scattering amplitudes and their differences to the slope of the phase shift. By definition, the signal I i is the raw current acquired by the ith detector segment normalized by the total current of the incident beam. The segments should be aligned with the scan direction in the sample plane, which may be rotated with respect to the scan direction seen in the detector plane in imaging mode due to the spiral electron trajectory in the projection system. After the rotation transformation, quadrant segments 1 and 2 are assumed to be located in the x>0 half-plane with respect to the sample scan, and the four segments are labeled counterclockwise.

[0115] The sum of the differential phase contrast (DPC) and annular bright-field (ABF) signals is calculated from the normalized quadrant signals and the reciprocal lattice vector k (corresponding to the extent of the bright-field illumination cone) according toBF can be found from. ABF=I1+I2+I3+I4, TIFF2025507253000017.tif12170, TIFF2025507253000018.tif12170

[0116] The normalization of the DPC components by the total signal is a small accommodation for the loss of intensity due to scattering. Approximately, the DPC is related to the sample phase (φ) delay according to: TIFF2025507253000019.tif16170 and TIFF2025507253000020.tif17170

[0117] Quadrant detectors are commonly used for laser alignment or for measuring tip displacements, for example in atomic force microscopy. Unlike a Gaussian beam of a laser, STEM illumination projects a uniform diffractive disk with sharp edges, about which the sensitivity of a quadrant detector to displacements differs in cubic terms

[14] .

[0118] In the following, a semi-analytical approach is illustrated that allows accurate calculation of the diffraction pattern displacement due to a quadrant detector in a STEM mimicking a suitable position sensitive detector (PSD).

[0119] Assuming that the diffraction disk is in focus and uniform, the signal from the detector is proportional to the illuminated area in each quadrant. Adjacent quadrants form hemi-planes, and the area A of intersection between a circular spot and the hemi-plane is analytically determined by the radius R of the spot and the central angle θ measured at the circle center between the vertices of the hemi-plane line cutting the circle, as follows: TIFF2025507253000021.tif15170

[0120] In FIG. 10, the relationship is, for example, an angle θ Band the area of ​​intersection between the diffraction disk and quadrants 1 and 4. The area of ​​the i-th quadrant detector illuminated by the diffraction disk is the area of ​​the acquired signal I i is proportional to.

[0121] Therefore, the following expression is obtained for the ratio of the beam intensities G falling on the respective opposite halves: TIFF2025507253000022.tif17170TIFF2025507253000023.tif17170

[0122] Use Newton's method with up to 10 iterations to find the angle θ A and θ B can be determined accurately and quickly. For example, θ A[0] =πG A Starting from +π, TIFF2025507253000024.tif17170

[0123] As can be seen in FIG. 10, the isosceles triangle formed by the circle center and the vertices of the circle with the x-axis and y-axis provides the following: TIFF2025507253000025.tif17170TIFF2025507253000026.tif16170

[0124] From these equations, the central location x in terms of angle and radius c ,y c It is easy to express the sign of the coordinate as G A and G B It is taken from the sign of sinθ with π as the center. A and cosθ A Expanding the analytical formula TIFF2025507253000027.tif15170 and Provide TIFF2025507253000028.tif16170, This indicates the absence of orthogonal terms. The illumination cone in k-space is As in TIFF2025507253000029.tif11170, it corresponds to the radius R of the diffraction disk, the camera length L (which is the distance of the detector from the sample plane), and the wavelength λ.

[0125] Nearly constant phase gradient Fourier transform characteristics with TIFF2025507253000030.tif12170 Based on TIFF2025507253000031.tif10170, The diffracted beam is uniform and has a vector It appears to be shifted along the x-axis according to TIFF2025507253000032.tif7170.

[0126] The known linear approximation of the DPC, i.e., the phase gradient, can be reproduced as follows: TIFF2025507253000033.tif15170 and TIFF2025507253000034.tif15170

[0127] Therefore, the signal G A and signal G B DPC x Signaling and DPC y The calculated center location provides a direct measure of the intensity "center of mass" (COM) displacement for the thin specimen at the focal point.

[0128] In focus, Note that the TIFF2025507253000035.tif7170 term relates to a third-order correction of the location of the diffraction disk. This results in the quadratic term TIFF2025507253000036.tif7170 remains the primary contribution with no disk location involved.

[0129] The "differential phase contrast" DPC evaluation assumes that the detector segment response is uniform. This is an advantageous property of diode-based detectors (such as those commercially available from Opal). Figure 11 illustrates histograms of the response of such a detector in the different channels Ch1-Ch6. The insert in the figure shows the uniformity in the response as a focused probe is scanned over the sensitive area using the direct scanning mode. The histograms of the reported intensity in each channel are found in the main plot. The variability of the average response between segments is less than 5%. The DPC calculation is also valid if the quadrant segment is part of an annulus rather than a disk, as long as the spot completely covers the inner diameter of the annulus and does not extend beyond the outer diameter. The microscope diffraction camera length is chosen to achieve these conditions for the detector segments Ch1-4.

[0130] The operation of the quadrant detector as a position sensitive detector (PSD) was tested by manually steering the beam using the diffraction alignment controls and then comparing the responses. The diffraction disk was manually steered using the microscope controls for diffraction alignment and the x,y readouts from the system status panel were recorded along with the average signals of the four segments extracted from the spiral scan. Based on the four average signals, the position of the diffraction disk was calculated, as detailed above.

[0131] Reference is made to Fig. 12A and Fig. 12B, which show a comparison between the diffraction disk position based on the four-quadrant analysis (solid lines) and the diffraction disk position based on the microscope diffraction shift status readout (dashed lines) in two cases where the beam is statically deflected using the control panel, without a sample and with lens renormalization after every measurement (Fig. 12A) and with a replica grating sample inserted (Fig. 12B). In the first case (Fig. 12A), the probe was measured without a sample and each measurement was performed 3 seconds after lens renormalization. This proves the close correspondence. Note that the "status" values ​​represent the control voltages and the difference signal is the real measurement. In the second case (Fig. 12B), the beam displacement was measured through the replica grating sample. This proves that when the beam is displaced, the signal remains well within range and therefore the error signal can be used during the scan as a measure of the COM displacement.

[0132] Phase images of the original / raw scan data set, i.e. integrated differential phase contrast data (iDPC), were further calculated based on the PSD signals, i.e. directly from the evaluation of the quadrant signals as position sensitive detectors. These images are shown in Fig. 13A and Fig. 13B, where Fig. 13A shows an iDPC image based on first and third order terms obtained by analyzing the diffraction disk displacement at focus, and Fig. 13B shows an iDPC image based on first and third order terms obtained by analyzing the diffraction disk displacement at -1.4 μm defocus. It should be noted that the very prominent Bragg spot in the detector signal is almost completely removed in the analysis of the disk displacement.

[0133] The inventors have further demonstrated the ability of the data analysis technique of the present invention for contrast enhancement by comparison and combination of multiple simultaneously acquired detector segment signals. As a specimen, a non-planar net of boron nitride (BN) nanotubes was used, scanned with a 2048 x 2048 pixel pseudo-spiral pattern for 20 seconds at a probe half convergence angle of 3.7 mrad. The camera length (calibrated to 1500 mm) was chosen to almost fill the inner quadrant segments A-D without overlapping with the neighboring annular segment E, which then collects the dark field signal (see detector in Figure 7A). The acceleration voltage was 200 kV, and thus the FWHM probe diameter was about 0.5 nm, which was thin enough to show the average field phase gradient, rather than the steep footprints of individual atoms. In addition to the mass thickness contrast, there are discrete points of Bragg scattering coming from the unresolved lattice of the layered material.

[0134] Respective image data showing transmission images of boron nitride nanotubes calculated from quadrant (segments A-D) and ADF (segment E) data are presented in Figures 14A-D, where Figure 14A shows the iDPC phase shift by Fourier analysis (iDPC:FT), Figure 14B shows the iDPC phase shift by real space integration (iDPC:RS), Figure 14C shows the ADF scan (ADF), and Figure 14D shows the difference image from the sum of segments A-D at two defocus settings (Δf).

[0135] The iDPC image is shown in

[11] . TIFF2025507253000037.tif12170, where the Fourier transform F and the reciprocal lattice vector TIFF2025507253000038.tif9170 is specified in 2D. Therefore, the iDPC image in Figure 14A is k·F k {DPC} / k 2 is obtained from the imaginary part of the inverse transform of

[0136] 0.01k to remove the lowest spatial frequencies that suffer from insufficient signal-to-noise ratio BF An additional Gaussian high-pass filter at can be used

[10] . For the relevant iCOM, a Fourier integration method minimizes the noise contribution to the measurement of the stored field

[11] .

[0137] An alternative route to obtain an iDPC is through integration in real space, i.e. The file is TIFF2025507253000039.tif12170.

[0138] In particular, for DPC measurements, the vector fields are not themselves strictly conservative. Therefore, a detailed description of the real-space integration implemented in a code called intgrad2

[12] proves useful. The code uses the backslash linear equation solver in Matlab to solve for Nx*Ny variables. Solving a set of 2*Nx*Ny equations with TIFF2025507253000040.tif7170. Hence, we have a non-conserved vector field, i.e. For TIFF2025507253000041.tif17170, the solution to the conflicting gradients is obtained in a least-squares fashion. This code was used to obtain the iDPC image shown in Figure 14B (with the same high-pass filter as in Figure 14A).

[0139] In general, the differences in image detail revealed by the two integration methods are small compared to the ADF image of FIG. 14C acquired from segment E. Both integration methods appear to be consistent in sign of phase with respect to zero mean, and contrast is certainly present at low spatial frequencies, unlike the situation for phase-contrast TEM. We find real-space integration to be slightly preferable, as objects appear more uniform with less ringing.

[0140] In FIG. 14D, phase contrast is extracted in a manner similar to the phase-shift extraction in TEM images described in

[13] . The image was derived from the difference between one image in focus and another image at −1.4 μm defocus. Empirically, the inventors determined that k p It was found that a calculation in reciprocal space of the image using a power of 1 (rather than a power of 2 as in the TEM method) renders the richest detail, i.e. TIFF2025507253000042.tif17170

[0141] k in STEM compared to TEM p The difference in powers of is the wave function This can be explained using TIFF2025507253000043.tif6170 and the Transport of Intensity Equation notation, which reads as follows: TIFF2025507253000044.tif17170

[0142] The second term on the right hand side is neglected in TEM images, whereas in STEM it dominates. In detail, the results can be explained based on simulations of the effective contrast transfer function (CTF) for an ABF detector

[11] .

[0143] The difference CTF(Δz)-CTF(0) for small defocus Δz depends linearly on spatial frequency in the low range. Therefore, the formula TIFF2025507253000045.tif16170 is approximately proportional to the Fourier transform of the phase, and therefore k p The best image between the powers of is rendered in real space.

[0144] The technique of the present invention takes into account the parallax effect manifested in image shift due to defocus. When comparing images from four quadrant segments, a defocus-dependent lateral shift was observed. This is a wave phenomenon in bright field; focus-dependent image shift is not observed when the quadrant detector collects in dark field.

[0145] According to the invention (in either the electron beam scanning or optical scanning implementations of the invention), for images acquired away from perfect focus, the shifted images of all detector channels can be realigned ("deshifted") to compensate for parallax. This is illustrated in Fig. 15A-F for defocus settings of -1.5 μm, -0.5 μm and 1.5 μm for STEM. Here, the images in Fig. 15A-C correspond to a simple annular bright field (ABF) detector readout by directly summing the signals from the four channels A, B, C, D. The defocus blur is severe compared to the ADF image acquired simultaneously on segment E. The images in Fig. 15D-F correspond to the same scans summed after parallax correction by aligning the images by cross-correlation. The resulting image is clearly sharper than a simple sum of the images. For tilted samples, the defocus aberrations can be corrected separately in the two halves of the image.

[0146] The data analysis technique of the present invention utilizes compensation of parallax shifts to provide extended depth of field and suppress contrast inversion in phase images, which can be better explained by testing some implications of contrast transfer function (CTF) theory, which is detailed below.

[0147] The mathematical description of the scanning signals from a thick sample is that each of them has a phase delay This can be described in the unperturbed probe model following [1] as an incoherent superposition of independent contributions from a thin slice along the transmission direction, denoted by the subscript l, inducing a TIFF2025507253000046.tif7170. The theory relies on the Born approximation throughout the sample, rather than an explicit weak phase approximation, and thus relates the refractive index n to the phase retardation in each layer as follows: TIFF2025507253000047.tif16170 where Δz is the layer thickness and λ is the wavelength of the electrons. In two-dimensional Fourier space, the relationship between the scanning signal and the phase shift can be written for iDPC as follows

[11] : TIFF2025507253000048.tif10170 where each part depends linearly on the sample features via the contrast transfer function, which is the function of the defocus lΔz and spatial frequency k p Depends on. TIFF2025507253000049.tif17170TIFF2025507253000050.tif16170TIFF2025507253000051.tif15170 Third-order correction term TIFF2025507253000052.tif7170 involves a cube phase delay and can therefore be ignored in practice.

[0148] For ADF detectors, TIFF2025507253000053.tif16170, Here, ψ in denotes the incident probe wavefunction. The prefactor c is assumed to be constant in a particular setup, and the ADF CTF is It can be defined as TIFF2025507253000054.tif11170.

[0149] Understanding the role of bright-field parallax in iDPC is important for its analysis for scanning microscopy imaging of thick samples. We calculated various CTFs based on the theory of

[11] at various defocus and spatial frequencies.

[0150] The results show the calculation of CTF vs. defocus at three spatial frequencies, and the CTF for a particular quadrant detector (FIG. 7B), with other parameters as described in the text. iS (Figure 16A) and the CTF for a hypothetical quadrant detector without a central hole. iS (Figure 16B) and CTF under similar conditions w2 16A-16D, showing the CTF (modulo prefactor) of ADF (FIG. 16C) and the CTF of ADF (FIG. 16D).

[0151] The modeled detector geometry was chosen according to the dimensions of the detector in FIG. 7B, available from OPAL, and the details of the setup used in a Tecnai T20-F microscope, i.e., spherical aberration C s = 2 mm, condenser C2 aperture 30 μm, camera length L = 1500 mm, and wavelength λ = 2.5 pm. At a corresponding half convergence angle of 3.7 mrad, for a depth of field of 180 nm, the contribution of spherical aberration is practically negligible, which renders the CTF function either symmetric or antisymmetric with respect to defocus. CTF related to first order terms in iDPC iS And, CTF ADF are symmetric and sinc-like, and both decay similarly with increasing spatial frequency. Therefore, the first term in iDPC is expected to be similar to iCOM, since iCOM is defined as a function of the cross-correlation between probe intensity and phase delay.

[0152] In Figure 16A, the CTF iS is calculated for the above detector in Fig. 7B with the hole in the center, and in Fig. 16B we see the hypothetical case without the hole. Clearly, the hole introduces a zero crossing into the CTF, but the crossing is absent for out-of-focus values ​​smaller than the canonical depth of field defined by the convergence angle and wavelength.

[0153] Quadratic terms in iDPC TIFF2025507253000055.tif6170 is antisymmetric with respect to defocus, as can be seen in FIG. 16C. TIFF2025507253000056.tif6170 is counted on the out-of-focus axis, CTF ADF It has as many extrema as zero crossings in . Extremes and CTF in TIFF2025507253000057.tif6170 ADF The defocus locations of the zero crossings in are nearly identical at high spatial frequencies. Thus, the second iDPC term provides complementary information to that of the ADF. Most importantly, The occurrence of a linear relationship in TIFF2025507253000058.tif7170. Centered on the origin The slope of TIFF2025507253000059.tif6170 increases in value until it reaches an asymptote at the lowest spatial frequencies. The further away from the focus, the stronger the low-frequency intensity becomes due to the sign reversal around the focus. This strongly suggests the image shift due to parallax explained above.

[0154] In Figures 17A-17H, we calculated the iDPC signal (by real space integration) from the data set shown in Figures 15A-15F as bright field images. Figures 17A-17D are calculated based on matched quadrant images, where the parallax contribution has been computationally compensated for (by "deshifting" them). 17E-17H show the second part calculated from the remainder of the iDPC signal. TIFF2025507253000061.tif7170 (iDPC2 in Figures 1 and 2), i.e. TIFF2025507253000062.tif7170. As can be clearly seen in the figures, separation of the iDPC portions reveals a constant portion related to phase or projected potentials (FIGS. 17A-17D) and a disparity-induced portion related to depth or defocus (FIGS. 17E-17H), where objects appear white in underfocus.

[0155] In the following, the formal relationship between disparity offset and iDPC CTF is detailed: The TIFF2025507253000063.tif7170 portion can be expanded as follows: TIFF2025507253000064.tif12170Therefore, it shows that it is dominated by the disparity contribution.

[0156] In geometric optics, the wavefront aberration of a condenser lens is e -iχ(k) causes an angular ray deflection and hence a ray displacement in the plane of the probe. TIFF2025507253000065.tif12170 results, where k is the spatial frequency in the diffraction plane, and therefore δ is generally a function of the off-axis position of the detector elements. Assuming only the defocus ΔZ and no other lens aberrations, δ = λΔZk Displacements are introduced into the scanned image depending on the accumulated signal on the detector plane. The detector sensitivity W(k) integrates the associated ray displacements to give an effective image shift relative to the aberration-free image: S = ∫d 2 kW(k)δ(k) / ∫d 2 kW(k) can be obtained.

[0157] The image shift S for a detector of uniform sensitivity over the hemi-plane for x>0 will be opposite in value compared to the image shift for a detector over the hemi-plane for x<0.

[0158] Therefore, there is a disparity offset between the images of the different quadrant detectors, and therefore the DPC is affected by the image shift contribution. However, it has a similar sensitivity to the COM sensor, i.e., W(k)=k x If we consider two halves with k, then the two halves will reveal the same S value. This means that the COM sensor is insensitive to focus-related parallax effects, as it is for uniform aberrations in k.

[0159] The inventors have demonstrated that the CTF of the second term of the iDPC is formally related to the image shift difference and therefore provides a depth contrast image of the sample.

[0160] The leading term in TIFF2025507253000066.tif7170 is the integrated CTF in reciprocal space related to the cosine of the phase contribution to the DPC vector. c Without any restrictions, the detector sensitivity W x Based on (k), only the x component is considered and similar results are obtained for the y component. According to

[11] , the calculation is The file is TIFF2025507253000067.tif12170. Use TIFF2025507253000068.tif10170, Uniform aberration χ(k)=χ(-k) and symmetrical condenser opening Assuming TIFF2025507253000069.tif9170, the calculation becomes a convolution term. TIFF2025507253000070.tif11170For small aberration phase shifts, the inventors make the following approximation, TIFF2025507253000071.tif16170 Therefore, The file is TIFF2025507253000072.tif22170.

[0161] Ignoring the convolution with the probe, the CTF can be integrated over the x and y components via: TIFF2025507253000073.tif17170

[0162] As k approaches 0, the convolution is replaced by an integration over k-space, and thus The file is TIFF2025507253000074.tif37170.

[0163] Written in this form, the inventors have We observed that the difference in the image shift between the left and right quadrants of the DPC detector is proportional to the sum of the image shifts between the upper and lower quadrants of the DPC detector. Thus, in the absence of lens aberrations, the difference in the image shift between the left and right quadrants of the DPC detector is proportional to the sum of the image shifts between the upper and lower quadrants of the DPC detector. TIFF2025507253000076.tif7170 is It is expected to read TIFF2025507253000077.tif9170.

[0164] The above-described Figures 17A-17H demonstrate the principles of the invention that arise from the separate analysis of the various CTF contributions to iDPC presented above. In particular, the figures show the first part ( The contrast of the first part (TIFF2025507253000078.tif7170) is hardly affected by the defocus, while the contrast of the second part ( The first part reveals the inventor's understanding that the contrast transfer function (CTF) at focus is strongly affected. iS This is useful for tomography, since it is possible to deconvolve the image by TIFF2025507253000080.tif7170 provides instantaneous relative height information from a single scan. Observe that the objects in the TIFF2025507253000081.tif7170 image almost disappear in focus, in underfocus they are white, and in overfocus they are dark, and it is easy to see which of the nanotubes is on top based on the shade difference at Δf=0.5 μm, for example, as shown in FIG. 17F.

[0165] The combined images of Figs. 15A to 15C are not used, but Figs. 17A to 17D are used. In the TIFF2025507253000082.tif7170 image, a single significantly "bright" tip of a nanotube can be discerned where the image intensity extends beyond the material boundary. The same feature was not particularly prominent in the ABF image.

[0166] FIG. 18 shows the DPC vector field (inverted) for a portion of the data in FIG. 17A-H determined from the raw DPCx and DPCy components related to the probe-convolved phase gradient. The large magnitude arrows are interpreted as electric fields pointing towards the accumulated negative charge. At scales larger than the probe diameter, they reflect the local electric field along the scan direction. The arrow direction and magnitude indicate the electric field due to excess negative charge accumulated on the nanotube boundary and at the sharp tip, approximately aligned along the circular scan trajectory. Electron beam charging is a common complication in scanning electron microscopy and STEM, but the DPC vector representation provides a means to exploit and possibly quantify it. Together with the capability of precise and compensated scan patterns, there is the potential to monitor charging dynamics and possibly determine conductivity.

[0167] The techniques of the present invention may therefore advantageously be implemented in scanning microscopes, and in particular in standard S / TEMs, by using a multi-segment detector therein and applying the above-described data analysis of all detection channels.

[0168] Also, as explained above, scanning microscope operation can be improved by appropriate control of the above-described parameters and conditions of the scanning mode. The above-described scan generator allows for arbitrary scan patterns. The inventors have demonstrated scanning with minimal acceleration, as opposed to conventional raster scanning, and providing correction algorithms to account for the delay of the probe position relative to the drive signal. As a data collection system, there are seven channels with simultaneous acquisition. These features, combined with the use of segmented detectors, particularly diode detectors, improve STEM performance even further. Conventional brightfield and high-angle annular darkfield signals were also recorded. The digitization speed is sufficient for significant oversampling in time, which allows for effective interpolation from non-conventional scan patterns to a Cartesian grid of presentable images. The time stream can be saved for further analysis. The system was programmed for compatibility with the popular SerialEM software package for microscope control and easy integration with high performance workflows.

[0169] The capabilities of multi-channel recording have been explored in various combinations to generate contrast even from weakly scattering specimens (such as boron nitride nanotubes). Compensation of the out-of-focus image shift from off-axis detector elements provides a simple separation of phase and depth contrast in iDPC. Compared to iCOM, the deshifted iDPC method essentially provides depth information for free. It should be understood that the data analysis principles of the present invention are not limited to any number of multiple detection channels and any type of multi-segment detection used to provide a scanning data set. The data analysis system of the present invention is applied as post-acquisition and thus can be useful for analyzing data collected by any multi-channel detection.

Claims

1. A computer system for determining a structural image of a sample, the computer system being configured for data communication with a measurement data provider for receiving raw measurement data provided by a scanning microscope, the raw measurement data being indicative of at least one scan data set acquired in a scanning session, the scan data set (I Ｍ ) Ｎ wherein N measurements correspond to a sequence of N measurements on a sample located proximate a focal plane of the scanning microscope, each measurement comprising data provided by M detection channels associated with an M-segment detector (M≧3), and the computer system comprises a data analyzer configured and operable to process the at least one scan data set to compensate for image shifts induced by off-axis detection channels for sample features in out-of-focus planes, thereby obtaining data indicative of a parallax-corrected scan image of the sample, thereby enabling separation between phase and depth information and extraction of a depth-contrast image of the sample from a single scan data set.

2. The data analyzer analyzes the data indicative of the parallax corrected scanned image and the scanned data set (I) to extract the depth contrast image of the sample, the depth contrast image comprising signed intensities, light and dark, indicating degrees of defocus indicative of depths of features in the sample. Ｍ ) Ｎ and data indicative of an integrated differential phase agreement of.

3. The data analyzer includes: The scanning data set (I Ｍ ) Ｎ to obtain a deshifted scan data set (I １ ｄｓ , . . . I Ｍ ｄｓ ) Ｎ and further processing the deshifted scan data set to obtain corresponding deshifted differential phase contrast data DPC ｄｓ and thereby providing said data indicative of the parallax corrected scanned image of the sample.

3. A computer system according to claim 1 or 2, comprising:

4. The first data processor processes the deshifted differential phase contrast data DPC ｄｓ from the corresponding integrated differential phase contrast data iDPC １ The computer system of claim 3 , further configured to determine:

5. The data analyzer analyzes the raw scan data set (I Ｍ ) Ｎ Using data showing an integrated differential phase contrast image iDPC calculated from the iDPC and the iDPC １ 5. The computer system of claim 4, further comprising a second data processor adapted to determine a difference between

6. The first data processor, The scanning data set (I Ｍ ) Ｎ , an image I acquired by each i-th detection channel based on the N measurements along the scan ｉ (x,y) by mapping locations in the image according to the location of a focused scanning beam in a focal plane; The image I of the scan data set ｉ (x,y) to process at least some of the ｉ The image shift (Δx) between (x, y) and a hypothetical image corresponding to a central detection channel that can be collected by a central region of the detector on the optical axis defined by the illumination path of the microscope. ｉ , Δy ｉ ) determining , determining the opposite shift value and subtracting the image I provided by each i-th detection channel from the image I ｉ (x,y) to obtain a deshifted scan data set (I １ ｄｓ , . . . I Ｍ ｄｓ ) and For all the detection channels, the deshifted images (I １ ｄｓ (x, y)...I ｍ ｄｓ (x, y)) to obtain the differential phase contrast data DPC ｄｓ thereby compensating for out-of-focus image shifts induced by the off-axis detection channels and providing data indicative of a parallax-corrected scanned image of the sample. By performing Ｍ ) Ｎ 4. A computer system as claimed in claim 3, adapted to process:

7. The opposite shift values ​​are ｉ (x, y) Ｎ 7. The computer system of claim 6, wherein m is a mean shift obtained from measurements by each of the M detection channels along the scan, applied with opposite sign to m.

8. The data analyzer analyzes a deshifted image (I １ ｄｓ (x, y)...I ｍ ｄｓ 8. The computer system of claim 3, further configured to determine a sum of intensity signals of (x,y) to provide an extended depth of field.

9. The image shift (Δx ｉ , Δy ｉ ) for at least some of the pairs of detection channels i,j in at least some of the images of the scan data set. ｉ and signal I ｊ and performing an inter-segment cross-correlation between the image shift (Δx ｉ , Δy ｉ 9. The computer system of claim 6, further comprising: determining a first metric for the first pair of metric values;

10. The data analyzer performs a model-based analysis on the image shift (Δx ) using a predefined model that describes the relationship between image shift, defocus, and dominant aberrations. ｉ , Δy ｉ 10. The computer system of claim 6, further configured to determine the degree of defocus Δz by applying the Δz filter to the data.

11. 11. The computer system of claim 6, wherein the data analyzer is further configured and operable to utilize the image shift and provide a depth-labeled phase contrast image for each selected region of interest of the scanned image.

12. 12. The computer system of claim 1, wherein the raw measurement data is obtained from a scanning transmission electron microscope (STEM) utilizing the M-segment detector.

13. The computer system of claim 1 , wherein the raw measurement data is acquired by a scanning confocal microscope utilizing the M-segment detector.

14. A computer system according to claim 1 , configured for data communication with a storage device in which the raw measurement data is stored and provided.

15. 15. The computer system of claim 14, wherein the storage device is associated with a scanning microscope that performs the measurements.

16. A scanning microscope unit, comprising: a scanning unit for probing a sample surface with radiation; detecting said radiation transmitted through the sample; and acquiring a scan data set (I) corresponding to a sequence of N measurements on a sample located close to a focal plane of said scanning microscope acquired in a scanning session. Ｍ ) Ｎ a detection unit with an M-segment detector (M≧3) located in a Fourier plane relative to a focal plane of the scanning unit for generating measurement data including: A control unit comprising a computer system according to any one of claims 1 to 11; An imaging system comprising:

17. The imaging system of claim 16 , wherein the radiation is electron beam radiation.

18. The imaging system of claim 17 , wherein the scanning unit is configured to provide a raster scan.

19. The imaging system of claim 17 , wherein the scanning unit is configured to provide a quasi-spiral or circular scan pattern, thereby minimizing changes in probe acceleration.

20. 20. The imaging system of claim 17, wherein the detection unit comprises a six-segment detector structure, the six segments including four-quadrant detector segments arranged concentrically around an annular ring segment.

21. 21. The imaging system of claim 20, wherein the detection unit further comprises at least one of a bright field detector aligned with a central aperture area of ​​the six-segment detector structure and a high angle annular dark field (HAADF) detector.

22. The imaging system of claim 16 , wherein the radiation is optical radiation.

23. The imaging system of claim 22 , wherein the scanning microscope unit has a confocal configuration.

24. 24. The imaging system of claim 22 or 23, wherein the scanning unit is configured to provide a raster scan.

25. 1. A method for use in the reconstruction of a structural image of a sample, the method comprising: At least one scanning data set (I Ｍ ) Ｎ providing raw measurement data indicative of processing and analyzing the at least one scan data set to compensate for image shifts induced by off-axis detection channels for sample features in out-of-focus planes, thereby obtaining data indicative of a parallax-corrected scan image of the sample, which allows for separation between phase and depth information and extraction of a depth-contrast image of the sample from a single scan data set; A method comprising:

26. The data representative of the parallax-corrected scanned image and the scanned data set (I) are then compared to extract the depth-contrast image of the sample, the depth-contrast image comprising signed intensities, light and dark, indicating degrees of defocus corresponding to depths of features in the sample. Ｍ ) Ｎ and data indicative of an integrated differential phase agreement of.

27. The processing and analyzing step comprises: The scanning data set (I Ｍ ) Ｎ to obtain a deshifted scan data set (I) that is compensated for image shifts induced by the off-axis detection channels for sample features in the out-of-focus plane. １ ｄｓ , . . . I Ｍ ｄｓ ) determining Processing the deshifted scan data set to obtain corresponding deshifted differential phase contrast data DPC. ｄｓ thereby providing said data indicative of said parallax corrected scanned image of said sample.

27. The method of claim 25 or 26, comprising:

28. The deshifted differential phase contrast data DPC ｄｓ from the corresponding integrated differential phase contrast data iDPC １ 30. The method of claim 27, further comprising determining:

29. Raw scan data set (I Ｍ ) Ｎ Using data showing an integrated differential phase contrast image iDPC calculated from the iDPC and the iDPC １ 30. The method of claim 28, further comprising determining a difference between:

30. The scanning data set (I Ｍ ) Ｎ to obtain the deshifted scan data set (I １ ｄｓ , . . . I Ｍ ｄｓ ) Ｎ To determine In the scan data set, an image I is acquired by each i-th detection channel based on the N measurements along the scan. ｉ (x,y) by mapping locations in the image according to the location of a focused scanning beam in a focal plane; The image I of the scan data set ｉ (x, y) from an image corresponding to a central detection channel that can be collected by a central region of the detector that intersects an optical axis defined by an illumination path of the microscope. ｉ Image shift between (x, y) (Δx ｉ , Δy ｉ ) determining , determining the opposite shift value and subtracting the image I provided by each i-th detection channel from the image I ｉ (x,y) to obtain a deshifted scan data set (I １ ｄｓ , . . . I Ｍ ｄｓ ) and For all the detection channels, the deshifted images (I １ ｄｓ (x, y)...I ｍ ｄｓ (x, y)) to obtain the differential phase contrast data DPC ｄｓ thereby compensating for out-of-focus image shifts induced by the off-axis detection channels and providing data indicative of a parallax-corrected scanned image of the sample.

30. The method of any one of claims 27 to 29, comprising:

31. The processing and analyzing step comprises: obtaining a deshifted image (I １ ｄｓ (x, y)...I ｍ ｄｓ 31. The method of any one of claims 27 to 30, further comprising determining a sum of the intensity signals of (x, y)).

32. The image shift (Δx ｉ , Δy ｉ ) for at least some of the pairs of detection channels i, j in the at least some of the N measurements. ｉ and each I ｊ and performing an inter-segment cross-correlation between Δx ｉｊ and Δy ｉｊ and determining the image shift (Δx ｉ , Δy ｉ 32. The method of claim 30 or 31, comprising determining the

33. The processing and analyzing step performs a model-based analysis on the image shift (Δx ) using a predefined model that describes the relationship between image shift, defocus, and dominant aberrations. ｉ , Δy ｉ 33. A method according to any one of claims 30 to 32, comprising determining the degree of defocus Δz by applying a Δz filter to the data.

Images