Systems and methods for image segmentation from sparse particle impact data

Abstract

Systems and methods for segmenting an image are described, including impinging a substrate surface at each of a plurality of sensing locations with a particle beam, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate surface. An intensity value associated with post-impact particles resulting from the impinging is measured and a measured intensity is computed based on the intensity values of the sensing locations. For each of a plurality of estimated locations defining another subset of the region of interest, and based on at least one of the following corresponding to one or more locations proximal to the estimated location, a corresponding estimated intensity is computed. Each of the plurality of estimated locations is segmented based on the corresponding estimated intensity and each of the sensing locations is segmented based on the corresponding measured intensity to correspond to one of a plurality of features.

Description

[0001] Related applications

[0002] This application relates to and claims priority to Canadian patent application No. 3,078,085, filed April 9, 2020, entitled “SYSTEM AND METHOD FOR IMAGESEGMENTATION FROM SPARSE PARTICLE IMPINGEMENT DATA”. The disclosure of the said Canadian patent application is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to imaging techniques, and more specifically, to systems and methods for performing image segmentation. Background Technology

[0004] For many grating-based imaging applications, such as electron microscopy, the bottleneck to imaging throughput is often the long data acquisition time. A potential approach to improving image acquisition speed is to acquire partial data and / or images and employ computational algorithms to construct a more complete image. However, doing so with sparsely sampled data in a way that meets the necessary criteria of accuracy, spatial resolution, and low noise remains a challenge.

[0005] Image fusion offers a way to reduce noise in images while seeking to maintain desired image properties—such as high image contrast. For example, Milillo [MILILLO, T. et al., “Image fusion combining SEM and ToF-SIMS images,” Surface and Interface Analysis, 47, 371–376 (2015)] improved SEM images by combining SEM data with optical microscopy using computational algorithms. Similarly, Tarolli [TAROLLI, JG et al., “Improving secondary ion mass spectrometry image quality with image fusion,” Journal of the American Society for Mass Spectrometry, 25, 2154–2162 (2014)] demonstrated improved quality of secondary ion mass spectrometry images without sacrificing chemical specificity through image fusion with electron microscopy images of higher intensity and therefore higher spatial resolution.

[0006] In some applications, images can need to be segmented, where labels are assigned to different pixels of an image so that pixels with the same label share similar characteristics to, for example, separate different phases of an image. Various segmentation methods have been proposed for different applications. For example, SEM images of porous materials can be segmented based on an algorithm that is based on last occurrence detection and assignment of structures in the z-direction, followed by local thresholding, as presented by Salzer [SALZER, M., et al. “A two-stage approach to the segmentation of FIB-SEM images of highly porous materials.” Materials Characterization, 69, 115-126 (2012)]. Alternatively, biological samples - such as red blood cells - can be segmented using contour fitting and filtering, as shown by Vromen and McCane [VROMEN, J., MCCANE, B., “Red blood cell segmentation from SEM images,” 2009 24th International Conference Image and Vision Computing New Zealand, 44-49, IEEE (2009)]. While Sim [SIM, K., et al. “Canny optimization technology for electron microscope image colourization,” Journal of Microscopy, 232, 313-334 (2008)] employs a combination of Canny edge detection, optimization and supervised segmentation - which requires initial user input - and Kreshuk [KRESHUK, A., et al. “Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images,” PloS ONE, 6, e24899. (2011)] demonstrates segmentation by machine learning techniques - which require a training data set - these segmentation methods can be surprisingly time consuming for many applications.Another common segmentation method is a combination of thresholding, filtering, and region growing, such as the process described by Yang and Buenfeld [YANG, R., BUENFELD, N. "Binary segmentation of aggregate in SEM image analysis of concrete," Cement and Concrete Research 31, 437-441 (2001)].

[0007] This background information is presented to reveal information believed by the applicant to be of possible relevance to the present application. It is not intended to be, and is not to be construed as, a recognition of any state of the art or acceptance of any related items as prior art. SUMMARY

[0008] The following presents a simplified summary of the general inventive concept described herein to provide a basic understanding of some aspects of the present disclosure. This summary is not an extensive overview of the present disclosure. It is not intended to identify key or critical elements of embodiments of the present disclosure or to delineate the scope of the scope of the present disclosure as it is defined by the claims set forth below and equivalents thereof.

[0009] There is a need for systems and methods for image segmentation that overcome some of the shortcomings of known techniques or at least provide useful alternatives thereto. Some aspects of the present disclosure provide examples of such processes and systems.

[0010] According to one aspect, there is provided a method for segmenting a plurality of features of a substrate from sparse imaging data, the method comprising: impinging a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate surface; measuring, by each of at least two particle sensors, an intensity value associated with post-impact particles resulting from the impinging at each of the plurality of sensing locations; for each sensing location, calculating a measured intensity based on the intensity values for the sensing location; for each of a plurality of estimated locations defining a further subset of the region of interest, calculating a corresponding estimated intensity based on at least one of the following corresponding to one or more locations proximal to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segmenting each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features.

[0011] In some embodiments, the calculated measured intensity is further based on at least one of the following corresponding to one or more positions proximal to the sensing position: one or more proximal measured intensities, and one or more proximal estimated intensities.

[0012] In some embodiments, calculating the measured intensity and the corresponding estimated intensity comprises solving an optimization problem.

[0013] In some embodiments, the measured intensity is determined from a combination of intensity values measured by the two or more particle sensors.

[0014] In some embodiments, each estimated intensity is calculated based on the corresponding estimated position and a distance between each position of the one or more proximal measured intensities and one or more proximal estimated intensities used in calculating the estimated intensity.

[0015] In some embodiments, each of the estimated intensities is related to a noise characteristic of the measured intensity measured by one of the at least two particle sensors.

[0016] In some embodiments, the segmentation comprises calculating a greyscale value at each of the estimated positions.

[0017] In some embodiments, the segmentation comprises solving an optimization problem.

[0018] In some embodiments, the optimization problem comprises a penalty function.

[0019] In some embodiments, the penalty function is related to an expected intensity corresponding to each of the plurality of features of the substrate.

[0020] In some embodiments, the calculation of estimated intensities comprises an initial guess for segmenting estimated intensities to correspond to one of the plurality of features.

[0021] In some embodiments, a plurality of segmented images is generated, each of the plurality of segmented images corresponding to one designated layer of the substrate.

[0022] In some embodiments, the plurality of segmented images are vertically aligned to determine interconnections between them in the substrate.

[0023] In some embodiments, the method further comprises generating a segmented image that illustrates connectivity of components of the substrate.

[0024] In some embodiments, the plurality of sensing positions comprises one or more rows.

[0025] In some embodiments, the one or more rows are oriented in more than one direction.

[0026] In some embodiments, the orientation of the one or more rows relative to one or more of the substrate surface features is rotated.

[0027] In some embodiments, the plurality of sensing locations comprises a Lissajous pattern.

[0028] In some embodiments, the plurality of sensing locations comprises an optimized pattern.

[0029] In some embodiments, the measurements by each of the at least two particle sensors are performed simultaneously.

[0030] In some embodiments, the particle beam comprises an electron beam.

[0031] In some embodiments, the two or more particle sensors comprise two or more of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

[0032] In some embodiments, the post-impact particles comprise at least one of primary electrons and secondary electrons.

[0033] According to another aspect, there is provided a method for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the method comprising: impinging a substrate surface at each of a plurality of sensing locations using a particle beam, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate surface; measuring, using a particle sensor, at each of the plurality of sensing locations, an intensity value associated with post-impact particles resulting from the impinging; for each sensing location, computing a measured intensity based on the intensity value of the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, computing a corresponding estimated intensity based on at least one of the following corresponding to one or more locations proximal to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segmenting each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features.

[0034] In some embodiments, the computed measured intensities are further based on at least one of the following corresponding to one or more positions proximal to the sensing position: one or more proximal measured intensities, and one or more proximal estimated intensities.

[0035] In some embodiments, computing the measured intensities and the corresponding estimated intensities includes solving an optimization problem.

[0036] In some embodiments, each estimated intensity is computed based on a corresponding estimated position and a distance between each position of the one or more proximal measured intensities and one or more proximal estimated intensities used in computing the estimated intensity.

[0037] In some embodiments, the estimated intensities are computed based at least in part on a noise property of the measured intensities.

[0038] In some embodiments, the segmentation includes computing a grayscale value at each of the estimated positions.

[0039] In some embodiments, the segmentation includes solving an optimization problem.

[0040] In some embodiments, the optimization problem includes a penalty function.

[0041] In some embodiments, the penalty function relates to an expected intensity corresponding to each of the plurality of features of the substrate.

[0042] In some embodiments, the computing of estimated intensities includes an initial guess for the segmentation of estimated intensities to correspond to one of the plurality of features.

[0043] In some embodiments, the method further includes generating a plurality of segmented images, each of the plurality of segmented images corresponding to one designated layer of the substrate.

[0044] In some embodiments, the method further includes vertically aligning the plurality of segmented images to determine interconnections between them in the substrate.

[0045] In some embodiments, the method further includes generating a segmented image that illustrates connectivity of components of the substrate.

[0046] In some embodiments, the plurality of sensing positions includes one or more rows.

[0047] In some embodiments, the one or more rows are oriented in more than one direction.

[0048] In some embodiments, an orientation of the one or more rows relative to one or more of the substrate surface features is rotated.

[0049] In some embodiments, the plurality of sensing locations comprises a Lissajous pattern.

[0050] In some embodiments, the plurality of sensing locations comprises an optimized pattern.

[0051] In some embodiments, the particle beam comprises an electron beam.

[0052] In some embodiments, the particle sensor comprises one of an in-lens sensor, a backscattered electron sensor, a segmented backscattered detector, and an Everhart-Thornley detector.

[0053] In some embodiments, the post-impact particles comprise at least one of primary electrons and secondary electrons.

[0054] According to another aspect, there is provided a system for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the system comprising: a digital application operable to receive, as input, sparse imaging data relating to intensity values as measured by each of at least two particle sensors in association with post-impact particles resulting from impinging a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate, the digital application further operable to: for each sensing location, compute a measured intensity based on the intensity value of the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, compute a corresponding estimated intensity based on at least one of the following corresponding to one or more proximal locations to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segment each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features.

[0055] In some embodiments, the computation of the measured intensity is further based on at least one of the following corresponding to one or more proximal locations to the sensing location: one or more proximal measured intensities, and one or more proximal estimated intensities.

[0056] In some embodiments, computing the measured intensities and the corresponding estimated intensities comprises numerically solving an optimization problem.

[0057] In some embodiments, the measured intensities are determined from a combination of intensity values measured by the two or more particle sensors.

[0058] In some embodiments, each of the estimated intensities is calculated based on the corresponding estimated position and a distance between each position of the one or more proximal measured intensities and one or more proximal estimated intensities used in calculating the estimated intensity.

[0059] In some embodiments, each of the estimated intensities is related to a noise characteristic of the measured intensities measured by one of the at least two particle sensors.

[0060] In some embodiments, the digital application is operable to calculate a grayscale value at each of the estimated positions for segmentation.

[0061] In some embodiments, the digital application is operable to segment the estimated intensities via solving an optimization problem.

[0062] In some embodiments, the optimization problem includes a penalty function.

[0063] In some embodiments, the penalty function is related to an expected intensity corresponding to each of the plurality of features of the substrate.

[0064] In some embodiments, the calculation of estimated intensities includes an initial guess for segmenting estimated intensities to correspond to one of the plurality of features.

[0065] In some embodiments, the system is further operable to generate a plurality of segmented images, each of the plurality of segmented images corresponding to one designated layer of the substrate.

[0066] In some embodiments, the system is further operable to vertically align the plurality of segmented images to determine interconnections between them in the substrate.

[0067] In some embodiments, the system is further operable to generate a segmented image that illustrates connectivity of components of the substrate.

[0068] In some embodiments, the plurality of sensing positions includes one or more rows.

[0069] In some embodiments, the one or more rows are oriented in more than one direction.

[0070] In some embodiments, the one or more rows are rotated with respect to an orientation of one or more of the substrate surface features.

[0071] In some embodiments, the plurality of sensing locations comprises a Lissajous pattern.

[0072] In some embodiments, the plurality of sensing locations comprises an optimized pattern.

[0073] In some embodiments, the particle beam comprises an electron beam.

[0074] In some embodiments, the two or more particle sensors comprise two or more of an in-lens sensor, a backscattered electron sensor, a segmented backscattered detector, and an Everhart-Thornley detector.

[0075] In some embodiments, the post-impact particles comprise at least one of primary electrons and secondary electrons.

[0076] In some embodiments, the system further comprises an instrument operable to impinge the substrate with a particle beam and comprising the two or more particle sensors.

[0077] In some embodiments, the instrument is a scanning electron microscope, and a transmission electron microscope or a tunneling electron microscope.

[0078] According to another aspect, there is provided a system for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the system comprising: a digital application operable to receive, as input, sparse imaging data relating to intensity values as measured by a particle sensor, associated with post-impact particles resulting from impinging a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate, the digital application further operable to: for each sensing location, compute a measured intensity based on the intensity value of the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, compute a corresponding estimated intensity based on at least one of the following corresponding to one or more proximal locations to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segment each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features.

[0079] In some embodiments, computing a measured intensity is further based on at least one of the following corresponding to one or more proximal locations to the sensing location: one or more proximal measured intensities, and one or more proximal estimated intensities.

[0080] In some embodiments, calculating the measured intensities and the corresponding estimated intensities includes numerically solving an optimization problem.

[0081] In some embodiments, each estimated intensity is calculated based on a corresponding estimated position and a distance between each position of the one or more proximal measured intensities and one or more proximal estimated intensities used in calculating the estimated intensity.

[0082] In some embodiments, the estimated intensities are calculated based at least in part on a noise property of the measured intensities.

[0083] In some embodiments, the digital application is operable to segment the estimated intensities by calculating a grayscale value at each of the estimated positions.

[0084] In some embodiments, the digital application is operable to segment the estimated intensities by solving an optimization problem.

[0085] In some embodiments, the optimization problem includes a penalty function.

[0086] In some embodiments, the penalty function relates to an expected intensity corresponding to each of the plurality of features of the substrate.

[0087] In some embodiments, the calculation of estimated intensities includes an initial guess for segmenting estimated intensities to correspond to one of the plurality of features.

[0088] In some embodiments, the digital application is further operable to generate a plurality of segmented images, each of the plurality of segmented images corresponding to one designated layer of the substrate.

[0089] In some embodiments, the digital application is further operable to vertically align the plurality of segmented images to determine interconnections between them in the substrate.

[0090] In some embodiments, the system is further operable to generate a segmented image that illustrates connectivity of components of the substrate.

[0091] In some embodiments, the plurality of sensing positions includes one or more rows.

[0092] In some embodiments, the one or more rows are oriented in more than one direction.

[0093] In some embodiments, the one or more rows are rotated with respect to an orientation of one or more of the surface features.

[0094] In some embodiments, the plurality of sensing locations comprises a Lissajous pattern.

[0095] In some embodiments, the plurality of sensing locations comprises an optimized pattern.

[0096] In some embodiments, the particle beam comprises an electron beam.

[0097] In some embodiments, the particle sensor comprises one of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

[0098] In some embodiments, the post-impact particles comprise at least one of primary electrons and secondary electrons.

[0099] In some embodiments, the system further comprises an instrument operable to impact the substrate with a particle beam and comprising the particle sensor.

[0100] In some embodiments, the instrument is an electron microscope.

[0101] Other aspects, features, and / or advantages will become more apparent after reading the following non-limiting description of specific embodiments thereof, given by way of example only, with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0102] Some embodiments of the present disclosure will be provided by way of example only with reference to the attached drawings, in which:

[0103] Figure 1A and Figure 1B are images of a substrate segmented using two variants of thresholding;

[0104] Figure 2A is a SEM image of a substrate, and Figure 2B is an image of Figure 2A processed using an edge detection process;

[0105] Figure 3 is a schematic representation of a cross-section of a microchip according to various embodiments;

[0106] Figure 4A and Figure 4B are SEM images of a substrate acquired using a backscattered electron detector and an in-lens detector, respectively, according to various embodiments;

[0107] Figure 5A and Figure 5B are partial images of a substrate acquired using a backscattered electron detector and an in-lens detector, respectively, according to various embodiments, each acquired every four lines of scan;

[0108] Figure 6 is a plot of a function for weighting values of neighboring pixels in an optimization problem according to at least one of the multiple embodiments;

[0109] Figure 7A and Figure 7B are images of solutions of optimization problems seeded with partial images acquired by a backscattered electron detector and an in-lens detector, respectively, according to multiple embodiments;

[0110] Figure 8A , Figure 8B and Figure 8C are plots of sample noise, a fast Fourier transform of the noise, and a model corresponding to the noise from the sample, respectively, according to multiple embodiments;

[0111] Figure 9 is a plot of a function used in an optimization problem that can be solved for segmenting an image according to multiple embodiments;

[0112] Figure 10A is an image reconstructed from every four line scans of a substrate by solving an optimization problem according to multiple embodiments, and Figure 10B is a segmented image generated by solving an optimization problem seeded with an image of Figure 10A as an initial guess according to multiple embodiments;

[0113] Figure 11A is a backscattered electron image of a substrate according to multiple embodiments, Figure 11B and Figure 11C are partial images of every four line scans of a substrate having Figure 11A acquired by a backscattered electron detector and an in-lens detector, respectively, according to multiple embodiments, and Figure 11D is an image of a substrate reconstructed from the images in Figure 11B and Figure 11C according to multiple embodiments;

[0114] Figures 12A to 12D is a segmented image of a reconstructed image of Figure 11D as generated by solving an optimization problem using a corresponding set of parameters shown in the respective table;

[0115] Figure 13A is a backscattered electron image of a substrate according to multiple embodiments, Figure 13B and Figure 13C are partial images of every two line scans of a substrate having Figure 13A acquired by a backscattered electron detector and an in-lens detector, respectively, according to multiple embodiments, and Figure 13D is an image of a substrate reconstructed from the images inFigure 11B and Figure 11C an image of the substrate reconstructed from the images in

[0116] Figures 14A to 14D is a segmented image of the reconstructed image of Figure 13D as generated by solving an optimization problem using the parameter set shown in the corresponding respective table

[0117] Figure 15 is a schematic diagram of an exemplary process flow for image segmentation from partial image data from more than one detector according to at least one embodiment; and

[0118] Figure 16 is a schematic diagram of an exemplary process for generating a segmented image from partial images acquired from a detector according to a number of embodiments.

[0119] For simplicity and clarity, elements of some of the figures can be illustrated as having particular interactions, and the illustrations can not necessarily be to scale. For example, the dimensions of some of the elements in the figures can be exaggerated relative to other elements to help to improve understanding of the numerous embodiments presently disclosed. Also, common but well-understood elements that are useful in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these numerous embodiments of the present disclosure. DETAILED DESCRIPTION

[0120] Numerous embodiments and aspects of the present specification will be described with reference to details discussed below. The following description and drawings are illustrative of the specification and are not to be understood as limiting the specification. Numerous specific details are described to provide a thorough understanding of numerous embodiments of the present specification. However, in certain instances, well-known or conventional details are not described in order to provide a discussion without unnecessary obscuration of numerous embodiments of the present specification.

[0121] Numerous instruments and processes will be described below to provide illustrative examples of embodiments of the systems disclosed herein. The embodiments described below do not limit any claimed embodiment and any claimed embodiment can cover processes or instruments different from those processes or instruments described below. The claimed embodiments are not limited to instruments or processes having all of the features of any one or more of the instruments or processes described below or common to multiple or all of the instruments or processes described below. An instrument or process described below can not be an embodiment of any claimed subject matter.

[0122] Furthermore, numerous specific details are set forth in order to provide a thorough understanding of the implementations described herein. However, it will be apparent to one skilled in the relevant art that the implementations described herein can be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the implementations described herein.

[0123] In this specification, an element can be described as "configured to" perform a function or "configured for" such a function. Generally, an element that is configured to perform a function or configured for performing a function is enabled to perform that function or is adapted to perform that function or is operable to perform that function or is operative to perform that function or is otherwise enabled to perform that function.

[0124] It should be understood that, for the purposes of this specification, language of "at least one of X, Y, and Z" and "one or more of X, Y, and Z" can be construed as meaning any one of X, Y, and Z individually, X, Y, and Z collectively, or any combination of two or more items X, Y, and Z (e.g., XYZ, XY, YZ, ZZ, etc.). Similar logic can apply to two or more items in any occurrence of "at least one of... " and "one or more of... " language.

[0125] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0126] Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The phrase "in one embodiment" or "in at least one embodiment" as used herein does not necessarily refer to the same embodiment, though it may. Additionally, the phrase "in another embodiment" or "in some embodiments" as used herein does not necessarily refer to different embodiments, though it may. Thus, embodiments as described can readily combine to form further embodiments.

[0127] In addition, as used herein, the term "or" is the inclusive "or" operator and is equivalent to the term "and / or", unless the context clearly dictates otherwise. The term "based on" is not exclusive and allows for additional factors to be based on unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of "a," "an," and "the" include plural references unless the context clearly dictates otherwise. The meaning of "in" includes "in" and "on."

[0128] The term "comprising" as used herein will be understood to mean that the following list is not exhaustive and can or can not include, as the case can be, any other additional suitable item(s), such as one or more additional feature(s), component(s) and / or element(s).

[0129] According to different embodiments, the systems and methods described herein provide different examples in which images can be segmented using computational analysis algorithms. In some embodiments, such segmentation can be performed on images computed to be completed from sparse data sets and / or partial images using the processes and systems described below. While some examples described herein can refer to images or partial images acquired from scanning electron microscope data, those skilled in the art will appreciate that the systems and processes disclosed herein can also be applied to other imaging applications, such as those performing raster imaging of a sample, and / or those acquiring data through, for example, other ion beam, topography, or optical imaging platforms, non-limiting examples of which can include, but are not limited to, transmission electron microscopy (TEM), atomic force microscopy (AFM), confocal microscopy, etc.

[0130] As the process of raster imaging can be time consuming, collecting partial noisy images and using computational optimization to complete them can improve image acquisition speed. For example, and according to at least one of the multiple embodiments described herein, collecting every n rows of data according to an otherwise typical electron microscope imaging protocol can reduce acquisition time by a factor of 1 / n (e.g., acquiring every fourth row of data from a substrate using a SEM can reduce acquisition time by a factor of 1 / 4). Moreover, according to multiple embodiments, such computational optimization protocols can be tailored for additional applications. For example, if image segmentation is desired, the optimization problem to be solved computationally can include additional terms and / or variables related to so forth to be considered during optimization.

[0131] Various computational methods exist to address the problem of "filling" in portions of an image. However, the associated respective drawbacks limit their applicability to various situations. For example, classical image processing uses toolboxes for operating on sets of pixels that iterate over multiple single pixels or regions. These methods can use global information such as pixel value histograms in deciding what operation or what operations to perform or control the protocol's operation. Examples of such classical processing operations include filtering, interpolation, region shrinking, etc. Some methods can be complex and can draw on other methods that operate in frequency space in some instances, using geometric information including "snakes", etc. While such methods are often efficient in that each step is easily understood and thus more easily adjusted and debugged, a drawback of classical methods can be that each process can add and / or remove information that can be lost to subsequent processing steps. For example, an initial filtering that removes apparent noise can suppress data related to a feature of interest, resulting in a subsequent process that loses the ability to characterize the feature.

[0132] Model-based processes often include statistical models from which one extracts likely physical parameters that produced the observed data. Examples of model-based processes can include, but are not limited to, maximum likelihood estimation, Bayesian estimation, etc. One advantage of these models is that they can consider many or all of the relevant probabilities simultaneously, including measurement error, observed material properties, and geometric distribution, expected results, etc. Thus, one aspect of the model will often not be able to effectively erase information useful to another aspect, although practical computational methods will still likely have the same effect in the case of nonlinear models. In addition, another advantage of model-based methods is that one can, in many cases, use the same model to predict the accuracy of the output and even use this information to design more effective scanning techniques. On the other hand, model-based methods can require an expert to define the model and have a reasonable understanding of the mechanism.

[0133] Optimization protocols can also include machine learning methods. Such processes can use general-purpose models that have proven themselves in other applications, rather than starting with an understanding of the relevant physics and / or framework of the system. The resulting models can be extremely complex. However, end users often do not need to understand the results to apply them. One advantage is that they can be built quickly and do not require an understanding of statistics or optimization, and can outperform expert models in many cases, especially in the absence of a compelling model. However, drawbacks can include that such systems can often produce incorrect results, can be susceptible to distorted data, generally give no warning when they are used on data for which they were not trained, require large amounts of computation, and often require large amounts of human-curated data.

[0134] While various methods for image segmentation have been proposed, they can generally be application-specific, with variable performance and suitability based on the needs and substrate characteristics. For example, while thresholding (i.e., assigning labels to pixels of an image based on, e.g., a threshold intensity value) can be successfully employed on low-noise SEM images, it can perform poorly on images containing higher degrees of noise. An example of this is shown in Figure 1A and Figure 1B , Figure 1A and Figure 1B include SEM images of circuit boards or wires on integrated circuits. Figure 1A An example showing instances where thresholding can assign inappropriate labels to pixels due to noise (circles indicated by arrows). Similarly, attempts to remove noise, such as by applying a Gaussian filter, can result in reduced contrast, and / or cause separate parts of an image to appear connected, as shown in Figure 1B connected wire components of the SEM images of

[0135] Similarly, while Canny edge detection, used in conjunction with thresholding, can be suitable for imaging certain objects, such as red blood cells, this method can perform poorly, or can require too much user input and / or time, for segmenting many various layers of a PCB board or computer chip. An example of the deficiencies of this method is shown in FIG. 2, where Figure 2A a noisy SEM image results in poor Canny edge detection, the results of which are shown in Figure 2B .

[0136] Automated methods of image segmentation, and in particular automated methods that can be successfully employed in a short amount of time, have many potential values in applications such as reverse engineering. For example, integrated circuits used in computer-based systems often include many layers, and often have three distinguishable components: silicon with no metal, metal wires under the silicon layer, and metal vias (through-plane wires connecting different layers of the integrated circuit). Figure 3A schematic of one exemplary example of such a structure, which in this case has five metallization layers (the image license is Attribution 2.5 Generic under Creative Commons via https: / / commons.wikimedia.org / wiki / User:Cepheiden). To image such a sample, the upper layers can be cut off against a horizontal plane intersecting the vias, followed by polishing of the sample. Since the vias connect the wires in the upper and lower metallization layers, they can appear "inside" the wires in the resulting image. The metal of the vias can be exposed by such a process, and can result in the vias being very bright features of the produced image. It is therefore advantageous to be able to quickly acquire SEM images on a layer-by-layer basis, and to perform accurate segmentation of such images to reverse engineer the connections between various wire components.

[0137] While those skilled in the art will appreciate that the systems and methods described herein are applicable to a wide range of imaging applications, various exemplary embodiments of the present disclosure will now be provided in the context of integrated circuit images using SEM images and / or partial SEM images or data. A number of examples can include the use of various ion detectors or other grating-based imaging techniques. Non-limiting examples of such detectors can include backscattered electron (BSE) detectors, secondary electron (SE) detectors, Everhart-Thornley (E-T) detectors, through-the-lens (TTL) detectors, in-lens detectors, AFM, etc.

[0138] Referring to FIGS. 4-14, a number of embodiments utilizing partial images acquired from in-lens detectors and BSE detectors will now be described. According to some embodiments, the images or partial images can be collected simultaneously, such that the data from each respective detector is aligned. However, those skilled in the art will appreciate that the embodiments disclosed herein utilizing multiple partial images for segmentation purposes can also acquire the images consecutively, e.g., using a single detector. Moreover, aspects of the present disclosure can be applicable to systems in which a single partial image is acquired or, conversely, more than two images are acquired, while remaining within the scope of the present disclosure.

[0139] Figure 4A and Figure 4BSEM images of the same portion of a circuit chip using a backscattered electron detector (BSD) and an in-lens detector are shown. In this example, and according to at least one embodiment, the images are collected simultaneously using a detector that is oriented 30° with respect to the orientation of the sample part. According to some embodiments, such a configuration can be employed during acquisition of a portion of an image to maintain a high degree of information about the extent of the part being imaged.

[0140] As can be seen in Figure 4A and Figure 4B , the images from the different detectors can have different noise and signal contrast. The process of image fusion can allow for a combination of, for example, the respective strengths of the two images. For example, according to at least one embodiment, one can employ an algorithm to take advantage of the high contrast of the BSD image and the noise characteristics (e.g. white noise) of the in-lens detector image. In the following examples and descriptions, and according to a number of embodiments, the reconstructed image is to be used for segmentation according to the partial data of such an image. One example of such partial data is shown in Figure 5A and Figure 5B , where a typical raster scan in SEM is shown using every fourth row of the BSD and in-lens detector, respectively.

[0141] According to a number of embodiments, the method of image segmentation according to partial data, such as the method shown in Figure 5A or Figure 5B , can be related to solving an optimization problem. The following description starts with a simple example optimization problem and increases in complexity in steps. According to one embodiment, the first step can be to solve the following optimization problem.

[0142]

[0143] In this example, V i,j is the pixel of the reconstructed image at position (i,j), I in-lens,i,j is the corresponding pixel in the (partial) in-lens detector image, S is the set of pixel positions of the scan, W i',j' is a two-dimensional array of weights for the neighborhood comparison, and λ1 is a penalty parameter that determines the trade-off between the smoothness of the resulting image and the data. The regularization term ∑ i, j∑ (i′,j′)∈N W i′,j′ (V i+i′,j+j′ -V i,j ) 2 penalizes the difference between a pixel and its neighbors. According to some embodiments, and in several subsequent examples, neighbors up to a distance of 9 can be considered and can have Figure 6The function types illustrated in the middle, where the height of the surface plot is the weight of their difference from the value of the central pixel. However, those skilled in the art will appreciate that other penalty functions known in the art can be employed within the solving of the optimization problem without departing from the scope of the present disclosure.

[0144] According to various embodiments, the pixels V i,j — in addition to referring to pixel positions that are not measured (i.e. do not correspond to “measured positions” - also referred to herein as “estimated positions”) — they can additionally refer to pixel values that have been measured. In other words, in some embodiments, although a pixel position is associated with one measured value, this value can be modified and / or overwritten (also referred to herein as “estimated”) when solving the optimization problem.

[0145] According to various embodiments, Figure 7A and Figure 7B illustrate the results of solving the optimization problem described above for Figure 5A and Figure 5B respectively. The resulting images can be relatively smooth, with less pronounced noise. However, the image quality can still not be sufficient for segmentation applications. Therefore, and according to at least one embodiment, image fusion can be introduced into the optimization problem.

[0146] As mentioned above, different probes can have different sensing properties. For example, BSD images typically have higher contrast than in-lens images, but can have non-white noise, as illustrated in Figure 8A A Fourier transform of one row of this noise Figure 8B shows that the noise can appear as if it is low-pass filtered. If the noise is asymmetric, as it is in this example, in some cases it can be modeled as a decaying exponential. For example, and according to at least one embodiment, the noise of a BSD image can be modeled as the following expression.

[0147]

[0148] In some cases, the noise can be modeled as a Gaussian with a standard deviation of approximately 2, which can result in a shape in Figure 8C In this example, this function is referred to as B. According to some embodiments, the image generated in the optimization problem - as discussed above - can be filtered by convolving the image with B, which according to some embodiments is symbolically ∑ k B k V i+k，jThe difference between this convolution and the BSD image can therefore be noise with the same characteristics. According to some embodiments, such a convolution of the difference can be considered as a weighting of the expected spectrum of the noise, and can be added to the in-lens term to create an optimization problem that can perform image fusion by solving the following optimization problem.

[0149]

[0150] According to some embodiments, if the goal is to perform image segmentation on the completed image, a segmentation penalty term can also be introduced into the optimization problem. Figure 9 One non-limiting example of such a segmentation penalty term is shown. In this case, the penalty function is a sixth order polynomial formed by the product of three parabolas. While those skilled in the art will appreciate that a variety of other penalty functions can be employed, the use of such a term can be advantageous due to the simplicity of the inclusion of three minima. According to some embodiments, the minima can be adjusted in the penalty function to correspond to the values b, d, and v, which can be adjusted based on a particular application. Those skilled in the art will appreciate that such values can be found empirically for a given set of materials, for example, or can be solved theoretically, simulated, selected by some optimization process, etc. In the following example of reverse engineering of integrated circuits, these gray scale values correspond to the gray scale values of silicon, buried metallization, and vias. In some embodiments, these values can be chosen to average the pixel values, while in other embodiments they can differ from the average to produce a higher quality segmentation. For example, the following example embodiment employs values that are not necessarily related to the average normalized pixel value.

[0151] Accordingly, in some embodiments, an example penalty function for segmentation can have the following form.

[0152]

[0153] According to some embodiments, various segmentation optimization problems can be non-linear and can benefit from a solution algorithm that includes more than one step. In some embodiments, solving a first optimization problem can provide a solution from which an initial guess can be obtained for a second optimization problem. For example, a first optimization problem can provide an initial completed image guess from a partial data set or partial image, and a second optimization step can use this initial guess to perform segmentation. According to at least one embodiment, one example would be to first solve the following problem, which can be quadratic and insensitive to the initial guess. This problem can be referred to interchangeably throughout the disclosure as the "first problem," or the "first optimization problem," or the "first step."

[0154]

[0155] In some embodiments, this can be followed by a second step, which according to some embodiments can be sensitive to the initial guess, and can include a non-convex problem, a non-quadratic optimization problem, similar to the following system of equations, which can be referred to herein interchangeably as the "second problem", or the "segmentation problem", or the "second optimization problem", or the "second step".

[0156]

[0157] In the above-mentioned embodiments, the two parameters λ1and λ2may be chosen or solved to control the weight and / or relative weight of the penalty function. For example, in some embodiments, increasing the parameter λ1may increase the smoothness (or blurriness) of the resulting image from the first optimization step. If increased to a significant degree, this can even result in all pixels being assigned to a single segment after the second step is completed. On the other hand, increasing λ2may increase the attractiveness of the pixel values to the assigned segment values (i.e. b, d, v). In such a case, if λ2is too large, the solution after the above-mentioned second step can resemble a rounded version of the initial guess from the first step, reducing the relative importance of the smoothness penalty (i.e. the fit data term can be relatively ignored).

[0158] According to some embodiments, the segmentation problem can be solved in two broad optimization solving steps, as described above. However, in some embodiments, additional steps can also be performed, which remain within the scope of the present disclosure. For example, it can be advantageous to normalize the pixel intensity values prior to solving the optimization problem. For example, according to at least one embodiment, if both the BSD detector and the in-lens detector acquire partial images (e.g. every n rows on a sample), one can first normalize I BSD and I in-lens The pixel values are normalized to be within a certain range (e.g. between 0 and 2).

[0159] The results of the two broad optimization problems after normalizing the pixel intensities to be within the range [0, 2] are shown in FIG. 10. In this example, and according to multiple embodiments, every four rows of scans of an integrated circuit in a SEM from both the BSD detector and the in-lens detector are provided as input. According to at least one embodiment, Figure 10A The completed image according to every four rows of scans from the first optimization problem is shown, while Figure 10B The segmented image after the second optimization problem, seeded with the solution from the first step as the initial guess (i.e. the second step is given Figure 10Aas an initial guess). In this example, the second step is given the parameters b = 0, d = 1, v = 2, λ1= 40000, and λ2= 100000.

[0160] In some embodiments, such as in applications where images of integrated circuits or other electronic devices are segmented, connectivity between components can be more important in the output segmented image than the actual size or thickness of the individual components. For example, in the upper layers of a computer chip, the wires can be thicker, with more space between them, which can result in a higher quality segmentation as compared to the lower layers of the chip, where the components are more densely packed. For example, when the wires are in close proximity to each other, a partial SEM image or line scan can not produce a clear enough change in intensity between the wires to distinguish them (i.e., the gray value between the wires can not reach the expected value of b).

[0161] Figures 11A to 11D An example of an integrated circuit using dense packing of wires is shown, according to one embodiment. In this example, Figure 11A A BSD image of the sample is shown. Figure 11B and Figure 11C Partial images of a quarter of a line scan as acquired by a BSD probe and an in-lens probe are shown, respectively. The results of solving the first optimization problem described above are shown in Figure 11D .

[0162] The results of the second optimization step using the images shown in Figure 11D as an initial guess are shown in FIG. 12 for various parameter sets. According to one embodiment, the "default" parameter set can be those shown in the table of Figure 12A However, in this example, the false connections are shown in the resulting segmented image of Figure 12A According to various embodiments, the parameter sets can be adjusted, either manually or automatically by a user, to adjust the optimization for wire thickness, border smoothness, connectivity, etc. An example of such an embodiment is shown in Figures 12B to 12D For example, modifying the gray values associated with the background (e.g., silicon) and the wires (i.e., b and d, respectively, in this example) can adjust the apparent thickness and connectivity of the wires. In this example, Figure 12D a parameter set with relatively high background and wire gray values is shown, according to one embodiment, that results in a segmented image with fully resolved wires.

[0163] For certain image completion and / or segmentation applications, and according to various embodiments, the optimization solution can be improved by acquiring an increased amount of data points (e.g., every two or three lines instead of every four lines in a SEM raster scan, for example). FIG. 13 shows SEM imaging of the same sample as in FIG. 11, but in this case,Figure 13B and Figure 13C show the pixel intensities for every two rows instead of every four rows as shown in FIG. 11 for SEM imaging acquired from the BSD detector and the in-lens detector, respectively. The completed image from the first optimization step is shown in Figure 13D . The segmented image from solving the second optimization problem using different segmentation parameters is shown in Figures 14A to 14D , using the completed image of Figure 13D as the initial guess. As in the previous embodiment, this example, which starts with every two rows of scanning from the BSD detector and the in-lens detector, also uses the parameter set shown in Figure 14D to provide a segmented image with fully resolved wires.

[0164] The various non-limiting embodiments described in FIGS. 4-14 are further schematically illustrated in Figure 15 . In this example process diagram, two detectors 1502 and 1504 acquire partial image data that is utilized by a first optimization problem algorithm 1510 to generate a more complete image. Example functions in the problem 1510 can include, but are not limited to, padding 1514, image fusion 1512, etc. According to various embodiments, the solution to the first problem 1510 can provide an image 1520 that can be used as an initial guess for a subsequent second optimization problem 1530. Non-limiting examples of functions that can optionally be included in the second problem 1530 are those initially included in the first problem 1510, such as an image fusion term 1532 or an image padding / adjustment term 1534, as well as terms and / or parameters related to image segmentation 1536. According to various embodiments, the solution to the second optimization problem can include a segmented image 1540.

[0165] Various embodiments of the present disclosure relate to optimization models for estimating segmented images based on partial image data. In yet other embodiments, full image data can be used with various segmentation algorithms disclosed herein, or images completed from partial data can be used in similar segmentation algorithms. In some embodiments, (partial) image data can be acquired in one or more SEM detectors, non-limiting examples of which can include a backscattered electron detector and / or a secondary electron detector. In some embodiments, a single detector can be used to acquire multiple data sets of the same area of a sample. For example, one embodiment can include a BSD detector that subsequently acquires data from the same row location of a sample at different angles, from which, similar to the optimization algorithms disclosed, optimization algorithms can perform image fusion, completion, and segmentation. In other embodiments, more than one of the same type of detector can be employed for sensing, either in parallel (simultaneously) or sequentially, or various combinations of detectors can be employed (e.g., two BSE detectors and one SE detector). In other embodiments, a single detector (e.g., BSD) can be used to sparsely sample data from a surface to be imaged, and then the sparsely sampled data can be used in optimization solutions as described herein.

[0166] As such, according to various embodiments, in Figure 15 various elements represented in FIG. 16 can be removed, altered, or added. In one embodiment, a detector, such as a SEM detector, can sparsely sample a substrate surface to obtain a partial image 1610. This partial image can be used as input to a function or series of functions that can provide a higher degree of resolution by, for example, filling in via an optimization problem in element 1620. Simultaneously or subsequently, the problem can be processed algorithmically, iteratively, or otherwise numerically to provide an output image 1640 that is segmented into a specified number of components, as in step 1630. For example, a microchip can be partially imaged in 1610 using a BSD detector, and an output segmented image at 1640 that includes three colors corresponding to silicon, wires, and vias.

[0167] Yet other embodiments can also employ various scanning patterns to obtain partial or complete data sets and / or images. For example, while the above-mentioned examples present embodiments in which data is utilized in an optimization problem of parallel rows of SEM scans according to a sample, other embodiments can include, but are not limited to, performing row scans in alternating directions (e.g., in the positive x-direction, then in the negative x-direction), scanning every n rows, first horizontally partitioning a distance y, then performing vertical row scans spaced x apart, scanning in a Lissajous pattern, taking measurements in random sample areas (e.g., sampling in randomly selected areas comprising 10% of the sample surface), or other forms of sparse sampling or down-sampling, which, for example, can improve the resulting images and / or solutions to the optimization problems described herein, or reduce sampling time while providing adequate quality and noise properties of the resulting images.

[0168] In yet other embodiments, images can be constructed to have various dimensions. For example, while the above-mentioned embodiments describe segmenting two-dimensional surfaces, three-dimensional segmentation can also be performed using the methods and systems described herein. For example, acquiring data from the same sample area from different angles can be used to reconstruct a three-dimensional topology, for example using a "(de)shading" technique, various forms of which will be understood by those skilled in the art. In other embodiments, a three-dimensional topology can be recreated from a set of two-dimensional segmented images, for example by stacking the segmented two-dimensional images to reconstruct a three-dimensional substrate, such as a microchip.

[0169] In various embodiments, various regularization and cost functions can be applied in the optimization problem, examples of which can include, but are not limited to, squared-difference-of-neighbours regularization, n-th order polynomial, and / or L-curve parameter selection.

[0170] In some measurement regimes, data acquired from various probes can be a function of the energy involved in the measurement. For example, relatively higher beam energies in SEM measurements, such as those produced at higher operating voltages, can allow electrons to penetrate the sample to a higher degree. According to at least one embodiment, such data can provide information about the sample volume and / or interaction volume rather than just the sample surface. Similarly, in some embodiments where data relating only to the surface layer of the sample can be desired, the voltage can be lowered. In various embodiments, different measurement energies and electric fields can be employed to extract various data sets, which can be used individually or in combination in optimization problems, such as those disclosed herein. In various embodiments, such data provided by adjusting various sensing parameters, such as voltage, can be employed to construct two- and / or three-dimensional models of the sample, for example in reverse engineering applications.

[0171] In various other embodiments, the sampling pattern (e.g., rows) can be selected based on the relative orientation of the sample features. For example, in a circuit board or integrated circuit reverse engineering application, if the wires in the sample are oriented horizontally, the row pattern can be selected to be slightly skewed from horizontal (e.g., rotated from 3 degrees to 20 degrees from horizontal) to produce higher quality completed images and / or segmented images. Such skew angles can be arbitrarily selected, determined empirically based on sample properties, optimized algorithmically, etc. Similarly, for a given scan pattern, the sample can be rotated in some embodiments to produce a shift in the feature angle relative to the scan pattern or relative to a specified Cartesian coordinate axis.

[0172] Those skilled in the art will appreciate that while some of the embodiments described herein employ algorithms written in Python using Pyomo in conjunction with Ipopt, various computing languages, packages, operating systems, algorithms with improved computational speed, etc. can be employed in various embodiments without departing from the scope of the present disclosure.

[0173] While the present disclosure describes various embodiments for illustrative purposes, such description is not intended to be limited to such embodiments. Rather, the Applicant's teachings described and exemplified herein encompass a variety of alternatives, modifications, and equivalents, as will be appreciated by those skilled in the art. The overall scope of the Applicant's teachings is defined by the appended claims. No particular order of steps or stages in the methods or processes described in the present disclosure is intended or implied. In many cases, the order of process steps can be varied without changing the purpose, effect, or meaning of the methods described.

[0174] The information as set forth in the various portions of this disclosure is not intended to be taken narrowly. The disclosure herein fully supports combinations of the various attributes and features described throughout this disclosure. The scope of the disclosure is fully intended to cover all such combinations. The foregoing detailed description has set forth various embodiments of the devices and / or methods via the use of specific terminology. It is to be understood, however, that the description is not to be limited to the terminology so used by virtue of descriptions associated with specific embodiments. Rather, a person of ordinary skill in the art will recognize that other embodiments can be practiced with the claimed devices and / or methods without departing from the spirit or scope of the disclosure. In some cases, details have been presented in terms of process flow diagrams, flowcharts, operation steps or the like, which illustrate the operations according to the claims. These while being presented in a particular order and configured for being performed in a particular order are merely illustrative and are meant to clarify the embodiments. They are not meant to limit or restrict the scope of these embodiments to the particular sequences, orders or steps presented, but they are meant to include alternative arrangements and sequences.

Claims

1. A method for segmenting a plurality of features of a substrate from sparse imaging data, the method comprising: impinging a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate surface; measuring, by each of at least two particle sensors, an intensity value associated with post-impingement particles resulting from the impinging at each of the plurality of sensing locations; for each sensing location, computing a measured intensity based on the intensity value for the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, computing a corresponding estimated intensity based on at least one of the following corresponding to one or more locations proximal to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segmenting each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features, wherein each of the estimated intensities is related to a noise characteristic of the measured intensities measured by one of the at least two particle sensors.

2. The method of claim 1, wherein the computing a measured intensity is further based on at least one of the following corresponding to one or more locations proximal to the sensing location: one or more proximal measured intensities, and one or more proximal estimated intensities.

3. The method of any one of claim 1 or claim 2, wherein the computing a measured intensity and the computing a corresponding estimated intensity comprise solving an optimization problem.

4. The method of claim 1 or claim 2, wherein the measured intensities are determined from a combination of intensity values measured by the at least two particle sensors.

5. The method of claim 1 or claim 2, wherein each of the estimated intensities is computed based on a distance between the corresponding estimated location and each of the one or more proximal measured intensities and one or more proximal estimated intensities used in computing the estimated intensity.

6. The method of claim 1 or claim 2, wherein the segmenting comprises computing a gray value at each of the estimated locations.

7. The method of claim 1 or claim 2, wherein the segmenting comprises solving an optimization problem.

8. The method of claim 7, wherein the optimization problem comprises a penalty function.

9. The method of claim 8, wherein the penalty function is related to an expected intensity corresponding to each of the plurality of features of the substrate.

10. The method of claim 9, wherein the computing of an estimated intensity comprises an initial guess used to segment based on estimated intensities to correspond to one of the plurality of features.

11. The method of claim 1 or claim 2, further comprising generating a plurality of segmented images, each of the plurality of segmented images corresponding to a designated layer of the substrate.

12. The method of claim 11, further comprising: aligning the plurality of segmented images vertically to determine interconnections between them in the substrate.

13. The method of claim 1 or claim 2, further comprising generating a segmented image that illustrates connectivity of components of the substrate.

14. The method of claim 1 or claim 2, wherein the plurality of sensing locations comprises one or more rows.

15. The method of claim 14, wherein the one or more rows are oriented in more than one direction.

16. The method of claim 15, wherein the one or more rows are rotated relative to an orientation of one or more of the substrate surface features.

17. The method of claim 1 or claim 2, wherein the plurality of sensing locations comprises a Lissajous pattern.

18. The method of claim 1 or claim 2, wherein the plurality of sensing locations comprises an optimized pattern.

19. The method of claim 1 or claim 2, wherein the measurements made by each of the at least two particle sensors are performed simultaneously.

20. The method of claim 1 or claim 2, wherein the particle beam comprises an electron beam.

21. The method of claim 20, wherein the at least two particle sensors comprise two or more of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

22. The method of claim 21, wherein the post-impact particles comprise at least one of primary electrons and secondary electrons.

23. A method for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the method comprising: impacting a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate surface; measuring, using a particle sensor, an intensity value associated with post-impact particles resulting from the impacting at each of the plurality of sensing locations; for each sensing location, computing a measured intensity based on the intensity value for the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, computing a corresponding estimated intensity based on at least one of the following corresponding to one or more locations proximal to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segmenting each of the plurality of estimated locations based on the corresponding estimated intensity, and each of the sensing locations based on the corresponding measured intensity, to correspond to one of the plurality of features, wherein the estimated intensities are computed based at least in part on noise properties of the measured intensities.

24. The method of claim 23, wherein the computing the measured intensities is further based on at least one of the following corresponding to one or more locations proximal to the sensing locations: one or more proximal measured intensities, and one or more proximal estimated intensities.

25. The method of either of claim 23 or claim 24, wherein the computing the measured intensities and the computing the corresponding estimated intensities comprises solving an optimization problem.

26. The method of claim 23 or claim 24, wherein each of the estimated intensities is computed based on a corresponding estimated location and a distance between the one or more proximal measured intensities and one or more proximal estimated intensities used in computing the estimated intensity.

27. The method of claim 23 or claim 24, wherein the segmentation comprises computing a gray scale value at each of the estimated locations.

28. The method of claim 23 or claim 24, wherein the segmentation comprises solving an optimization problem.

29. The method of claim 28, wherein the optimization problem comprises a penalty function.

30. The method of claim 29, wherein the penalty function relates to an expected intensity corresponding to each of the plurality of features of the substrate.

31. The method of claim 30, wherein the computing of the estimated intensities comprises an initial guess used to segment based on estimated intensities to correspond to one of the plurality of features.

32. The method of claim 23 or claim 24, further comprising generating a plurality of segmented images, each of the plurality of segmented images corresponding to one designated layer of the substrate.

33. The method of claim 32, further comprising: vertically aligning the plurality of segmented images to determine interconnections between them in the substrate.

34. The method of claim 23 or claim 24, further comprising generating a segmented image that illustrates connectivity of components of the substrate.

35. The method of claim 23 or claim 24, wherein the plurality of sensing locations comprises one or more rows.

36. The method of claim 35, wherein the one or more rows are oriented in more than one direction.

37. The method of claim 36, wherein the one or more rows are rotated relative to an orientation of one or more of the substrate surface features.

38. The method of claim 23 or claim 24, wherein the plurality of sensing locations comprises a Lissajous pattern.

39. The method of claim 23 or claim 24, wherein the plurality of sensing locations comprises an optimized pattern.

40. The method of claim 23 or claim 24, wherein the particle beam comprises an electron beam.

41. The method of claim 40, wherein the particle sensor comprises one of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

42. The method of claim 41, wherein the post-impact particles comprise at least one of primary electrons and secondary electrons.

43. A system for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the system comprising: a digital application operable to receive, as input, sparse imaging data relating to intensity values measured by each of at least two particle sensors associated with post-impact particles resulting from impinging a substrate surface with a particle beam at each of a plurality of sensing locations, the plurality of sensing locations defining a subset of locations within a region of interest of the substrate, the digital application further operable to: for each sensing location, compute a measured intensity based on the intensity values for the sensing location; for each of a plurality of estimated locations defining another subset of the region of interest, compute a corresponding estimated intensity based on at least one of the following corresponding to one or more locations proximal to the estimated location: one or more proximal measured intensities, and one or more proximal estimated intensities; and segment each of the plurality of estimated locations based on the corresponding estimated intensity and each of the sensing locations based on the corresponding measured intensity to correspond to one of the plurality of features, wherein each of the estimated intensities relates to a noise characteristic of the measured intensities measured by one of the at least two particle sensors.

44. The system of claim 43, wherein the computation of the measured intensities is further based on at least one of the following corresponding to one or more locations proximal to the sensing location: one or more proximal measured intensities, and one or more proximal estimated intensities.

45. The system of any one of claim 43 or claim 44, wherein computing the measured intensities and the corresponding estimated intensities comprises numerically solving an optimization problem.

46. The system of claim 43 or claim 44, wherein the measured intensities are determined from a combination of intensity values measured by the at least two particle sensors.

47. The system of claim 43 or claim 44, wherein each of the estimated intensities is computed based on a distance between the corresponding estimated location and each of the one or more proximal measured intensities and one or more proximal estimated intensities used in computing the estimated intensity.

48. The system of claim 43 or claim 44, wherein the digital application is operable to compute a grayscale value at each of the estimated locations for segmentation.

49. The system of claim 43 or claim 44, wherein the digital application is operable to segment the estimated intensity via solving an optimization problem.

50. The system of claim 49, wherein the optimization problem comprises a penalty function.

51. The system of claim 50, wherein the penalty function relates to an expected intensity corresponding to each of the plurality of features of the substrate.

52. The system of claim 51, wherein the calculation of estimated intensity comprises an initial guess used to correspond to one of the plurality of features based on a segmentation of estimated intensity.

53. The system of claim 43 or claim 44, further operable to generate a plurality of segmented images, each of the plurality of segmented images corresponding to one designated layer of the substrate.

54. The system of claim 53, further operable to vertically align the plurality of segmented images to determine interconnections therebetween in the substrate.

55. The system of claim 43 or claim 44, further operable to generate a segmented image that illustrates connectivity of components of the substrate.

56. The system of claim 43 or claim 44, wherein the plurality of sensing locations comprises one or more rows.

57. The system of claim 56, wherein the one or more rows are oriented in more than one direction.

58. The system of claim 57, wherein the one or more rows are rotated with respect to an orientation of one or more of the substrate surface features.

59. The system of claim 43 or claim 44, wherein the plurality of sensing locations comprises a Lissajous pattern.

60. The system of claim 43 or claim 44, wherein the plurality of sensing locations comprises an optimized pattern.

61. The system of claim 43 or claim 44, wherein the particle beam comprises an electron beam.

62. The system of claim 61, wherein the at least two particle sensors comprise two or more of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

63. The system of claim 62, wherein the post-impact particles comprise at least one of primary electrons and secondary electrons.

64. The system of claim 43 or claim 44, further comprising an instrument operable to impinge the substrate with a particle beam and comprising the two or more particle sensors.

65. The system of claim 64, wherein the instrument is a scanning electron microscope, a transmission electron microscope, or a tunneling electron microscope.

66. A system for segmenting a plurality of features of a substrate from sparse imaging data, the substrate comprising a plurality of features, the system comprising: A digital application operable to receive, as input, sparse imaging data relating to intensity values measured by a particle sensor in association with post-impact particles resulting from impacts of a particle beam against a substrate surface at each of a plurality of sensing locations defining a subset of positions within a region of interest of the substrate, the digital application further operable to: for each sensing location, compute a measured intensity based on the intensity value for the sensing location; for each of a plurality of estimated positions defining another subset of the region of interest, compute a corresponding estimated intensity based on at least one of the following corresponding to one or more positions proximal to the estimated position: one or more proximal measured intensities, and one or more proximal estimated intensities; and segment each of the plurality of estimated positions based on the corresponding estimated intensity and each of the sensing locations based on the corresponding measured intensity to correspond to one of the plurality of features, wherein the estimated intensity is computed based at least in part on a noise property of the measured intensity.

67. The system of claim 66, wherein the computing a measured intensity is further based on at least one of the following corresponding to one or more positions proximal to the sensing location: one or more proximal measured intensities, and one or more proximal estimated intensities.

68. The system of any one of claim 66 or claim 67, wherein computing the measured intensity and the corresponding estimated intensity comprises numerically solving an optimization problem.

69. The system of claim 66 or claim 67, wherein each of the estimated intensities is computed based on a distance between the corresponding estimated position and each of the one or more proximal measured intensities and one or more proximal estimated intensities used in computing the estimated intensity.

70. The system of claim 66 or claim 67, wherein the digital application is operable to segment the estimated intensities by computing a grayscale value at each of the estimated positions.

71. The system of claim 66 or claim 67, wherein the digital application is operable to segment the estimated intensities by solving an optimization problem.

72. The system of claim 71, wherein the optimization problem comprises a penalty function.

73. The system of claim 72, wherein the penalty function relates to an expected intensity corresponding to each of the plurality of features of the substrate.

74. The system of claim 73, the computing of estimated intensities comprises an initial guess used to segment estimated intensities to correspond to one of the plurality of features.

75. The system of claim 66 or claim 67, wherein the digital application is further operable to generate a plurality of segmented images, each of the plurality of segmented images corresponding to a specified layer of the substrate.

76. The system of claim 75, wherein the digital application is further operable to align the plurality of segmented images vertically to determine interconnections therebetween in the substrate.

77. The system of claim 66 or claim 67, further operable to generate a segmented image that illustrates connectivity of components of the substrate.

78. The system of claim 66 or claim 67, wherein the plurality of sensing locations comprises one or more rows.

79. The system of claim 78, wherein the one or more rows are oriented in more than one direction.

80. The system of claim 79, wherein the one or more rows are rotated relative to an orientation of one or more of the surface features.

81. The system of claim 66 or claim 67, wherein the plurality of sensing locations comprises a Lissajous pattern.

82. The system of claim 66 or claim 67, wherein the plurality of sensing locations comprises an optimized pattern.

83. The system of claim 66 or claim 67, wherein the particle beam comprises an electron beam.

84. The system of claim 83, wherein the particle sensor comprises one of an in-lens sensor, a backscattered electron sensor, a segmented backscatter detector, and an Everhart-Thornley detector.

85. The system of claim 84, wherein the post-impact particles comprise at least one of primary electrons and secondary electrons.

86. The system of claim 66 or claim 67, further comprising an instrument operable to impinge the substrate with a particle beam and comprising the particle sensor.

87. The system of claim 86, wherein the instrument is an electron microscope.

Images