Charging artifact mitigation via scanning direction rotation
By capturing and aligning multiple images of specimens with varied rotational scan directions and averaging them, the method addresses the challenge of charging artifacts in charged particle microscopes, providing high-quality images without niche hardware or tuning.
Patent Information
- Application Number
- JP2025091557
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-06-04
- Filing Date
- 2025-06-02
- Publication Date
- 2025-12-16
AI Technical Summary
Charged particle microscopes often produce charging artifacts, such as streaks and distortions, in images of non-conductive specimens, which are difficult to mitigate with existing techniques that require niche hardware or time-consuming parameter tuning, limiting their generality across different microscopes and specimens.
Capture multiple images of a specimen using varied rotational scan directions, align and crop them to a target direction, and then average them to generate an aggregate image that reduces or eliminates charging artifacts.
The method effectively eliminates charging artifacts across various specimens and microscopes without requiring specific hardware or tuning, ensuring high-quality images with reduced artifacts.
Smart Images

Figure 2025183178000001_ABST
Abstract
Description
[Technical Field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of the filing date of U.S. Non-Provisional Patent Application No. 18 / 732,999, filed June 4, 2024, the disclosure of which is incorporated herein by reference in its entirety.
[0002] The present invention relates to the reduction of charging artifacts due to rotation of the scan direction. [Background technology]
[0003] Some specimens may be prone to charging artifacts when imaged with a charged particle microscope. Charging artifacts are undesirable. Summary of the Invention
[0004] The following presents a summary of the invention to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or critical elements or to delineate the scope of particular embodiments or the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. One or more embodiments described herein describe a device, system, computer-implemented method, apparatus, or computer program product for facilitating charging artifact mitigation through scan direction rotation.
[0005] In accordance with one or more embodiments, a system is provided. The system may include a non-transitory computer-readable memory capable of storing computer-executable components. The system may further include a processor operatively coupled to the non-transitory computer-readable memory and capable of executing the computer-executable components stored in the non-transitory computer-readable memory. In various embodiments, the computer-executable components may include an access component capable of accessing a charged particle microscope having a specimen mounted thereon. In various aspects, the computer-executable components may further include an aggregation component capable of generating an aggregate image of the specimen based on multiple images of the specimen captured by the charged particle microscope according to a target scan direction and multiple rotational scan directions.
[0006] According to one or more embodiments, a computer-implemented method is provided. In various embodiments, the computer-implemented method can include accessing, by a device operatively coupled to a processor, a charged particle microscope having a specimen mounted thereon. In various aspects, the computer-implemented method can include generating, by an apparatus, an aggregate image of the specimen based on a plurality of images of the specimen captured by the charged particle microscope according to a target scan direction and a plurality of rotational scan directions.
[0007] According to one or more embodiments, a computer program product is provided for facilitating mitigation of charging artifacts by rotating the scan direction. In various embodiments, the computer program product can include a non-transitory computer-readable memory having program instructions embodied therein. In various aspects, the program instructions are executable by a processor to cause the processor to access a scanning electron microscope having a specimen mounted thereon. In various cases, the program instructions are further executable to cause the processor to cause the scanning electron microscope to capture multiple images of the specimen according to a target scan direction and multiple rotational scan directions, respectively, the multiple images showing the respective charging artifacts. In various cases, the program instructions are further executable to cause the processor to correctly rotate the multiple images captured according to the multiple rotational scan directions so that the multiple images are aligned with the target scan direction. In various aspects, the program instructions are further executable to cause the processor, after the correct rotation, to average the multiple images to generate an aggregate image of the specimen, wherein the visibility of the charging artifact in the aggregate image can be lower than the individual visibility of the charging artifact in the multiple images. [Brief explanation of the drawings]
[0008] Various embodiments will be readily understood by the following detailed description taken in conjunction with the accompanying drawings. To facilitate this description, like numerals refer to like structural elements. The embodiments are illustrated in the drawings by way of example and not by way of limitation. The drawings are not necessarily drawn to scale.
[0009] [Figure 1] FIG. 1 illustrates an example non-limiting block diagram of a scientific instrument module according to various embodiments described herein. [Figure 2] 1 illustrates an example non-limiting flow diagram of a computer-implemented method according to various embodiments described herein. [Figure 3]FIG. 1 shows a block diagram of an example non-limiting system that facilitates charging artifact mitigation through scan direction rotation, according to one or more embodiments described herein. [Figure 4] FIG. 1 shows a block diagram of an example non-limiting system including multiple rotational scan directions and multiple captured images that facilitates mitigation of charging artifacts through rotation of the scan direction, according to one or more embodiments described herein. [Figure 5] 1 illustrates an example non-limiting flow diagram illustrating a method for implementing multiple rotational scan directions to capture multiple images according to one or more embodiments described herein. [Figure 6] 1 illustrates an example non-limiting flow diagram illustrating a method for implementing multiple rotational scan directions to capture multiple images according to one or more embodiments described herein. [Figure 7] 1 illustrates an example non-limiting flow diagram illustrating a method for implementing multiple rotational scan directions to capture multiple images according to one or more embodiments described herein. [Figure 8] 1 illustrates an example non-limiting flow diagram illustrating a method for implementing multiple rotational scan directions to capture multiple images according to one or more embodiments described herein. [Figure 9] FIG. 1 illustrates a block diagram of a non-limiting example system including multiple first edited images that facilitates charging artifact mitigation through rotation of scan direction, according to one or more embodiments described herein. [Figure 10] 1 shows an example of a non-limiting block diagram illustrating a method for obtaining a plurality of first edited images according to one or more embodiments described herein. [Figure 11] 1 shows an example of a non-limiting block diagram illustrating a method for obtaining a plurality of first edited images according to one or more embodiments described herein. [Figure 12] 1 shows an example of a non-limiting block diagram illustrating a method for obtaining a plurality of first edited images according to one or more embodiments described herein. [Figure 13]FIG. 1 illustrates a block diagram of a non-limiting example system including multiple second edited images that facilitates charging artifact mitigation through rotation of scan direction, according to one or more embodiments described herein. [Figure 14] 1 shows an example non-limiting block diagram illustrating a method for obtaining a plurality of second edited images according to one or more embodiments described herein. [Figure 15] 1 shows an example non-limiting block diagram illustrating a method for obtaining a plurality of second edited images according to one or more embodiments described herein. [Figure 16] 1 shows an example non-limiting block diagram illustrating a method for obtaining a plurality of second edited images according to one or more embodiments described herein. [Figure 17] FIG. 1 shows a block diagram of an example non-limiting system including aggregate images that facilitates charging artifact mitigation through rotation of scan direction, according to one or more embodiments described herein. [Figure 18] 1 shows an example of a non-limiting block diagram illustrating how an aggregate image may be obtained according to one or more embodiments described herein. [Figure 19] 10 provides non-limiting examples of experimental results according to one or more embodiments described herein. [Figure 20] 10 provides non-limiting examples of experimental results according to one or more embodiments described herein. [Figure 21] FIG. 1 illustrates an example, non-limiting block diagram of a graphical user interface that may be used in the performance of some or all of the methods or techniques disclosed herein, according to various embodiments described herein. [Figure 22] 1 illustrates an example non-limiting block diagram of a computing device capable of performing some or all of the methods or techniques disclosed herein, according to various embodiments described herein. [Figure 23]1 illustrates an example non-limiting block diagram of a scientific instrument support system capable of performing some or all of the methods or techniques disclosed herein, according to various embodiments described herein. [Figure 24] 1 illustrates a block diagram of an example non-limiting operating environment that can facilitate one or more embodiments described herein. [Figure 25] 1 illustrates an example of a network environment operable to execute various implementations described herein. [Figure 26] 1 illustrates an example of a dual beam microscope that can be implemented according to various embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION
[0010] The following detailed description is merely exemplary and is not intended to limit the embodiments or the application or uses of the embodiments, nor is there any intention to be bound by any expressed or implied information presented in the preceding Background or Summary sections or in the Detailed Description section.
[0011] One or more embodiments will now be described with reference to the drawings, wherein like numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more embodiments. It will be apparent, however, that in various instances, one or more embodiments may be practiced without these specific details.
[0012] Various operations may be described in order as multiple separate acts or operations as is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be construed as implying that these operations are necessarily order dependent. In particular, these operations may be performed in an order different from that presented. The operations described may be performed in an order different from the described embodiment. Various additional operations may be performed, or described operations may be omitted, in additional embodiments.
[0013] While some elements may be referred to in the singular (e.g., "processor"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processor may be implemented with different ones of the operations being performed by different processors. As used herein, the phrase "based on" should be understood to mean "based at least in part on," unless otherwise specified.
[0014] Charged particle microscopes (e.g., scanning electron microscopes (SEMs), dual-beam microscopes) can be any suitable computerized device capable of capturing or generating microscopic or nanoscale images of specimens in scientific, laboratory, research, or clinical operating environments. To facilitate the capture or generation of such images, charged particle microscopes can utilize complex configurations of operating components (e.g., ion sources, electron sources, optical lenses or apertures, optical plates or deflectors, columns, coils, heaters, coolers, fluid valves, fluid pumps, circuit switches, specimen stages), sensors (e.g., ion detectors, electron detectors, voltmeters, thermistors, potentiometers, manometers), or consumables (e.g., carrier fluids, calibrants, filters, reactive gases).
[0015] Some specimens may be susceptible to charging artifacts when imaged with a charged particle microscope. In particular, when a non-conductive or other poorly conductive specimen (e.g., a ceramic specimen) is scanned with a charged particle microscope, the specimen may become irregularly or non-uniformly charged. Such irregular or non-uniform charging may appear visually as pronounced or other highly noticeable streaks (e.g., light or dark streaks, bars, scratches, or other distortions) in scanned images of the specimen captured or generated by the charged particle microscope. Such streaks may be referred to as charging artifacts.
[0016] Charging artifacts are undesirable. Thus, charging artifacts can obscure the specimen as shown in the scan image, thereby preventing any downstream analysis (e.g., image classification, image segmentation, image regression) planned or desired to be performed on the scan image from being performed with confidence or reliability. In some cases, scan images containing some charging artifacts may be damaged, contaminated, corrupted, or otherwise deemed unusable.
[0017] A variety of existing techniques attempt to mitigate or eliminate charging artifacts. Some of these techniques require the implementation of niche microscope hardware, such as charge neutralizers or conductive specimen coaters. Others require sample-specific tuning of microscope parameters, such as adjusting scan speed, beam voltage, or detector settings. Unfortunately, these techniques are significantly limited in generality. In fact, many (if not most) charged particle microscopes in use in the field lack or are otherwise not equipped with the niche hardware that some existing techniques rely on to attenuate charging artifacts. Furthermore, there are countless different types and compositions of specimens that are prone to charging artifacts. Therefore, existing techniques that rely on tuning microscope parameters require time-consuming or effort-intensive fine-tuning or experimentation to identify which specific microscope parameter values (e.g., which specific scan speed, which specific beam voltage, which specific detector settings) should be used for any given specimen. In other words, different specimens may have different compositions and therefore require different microscope parameter values to avoid or reduce charging artifacts (e.g., a first microscope parameter configuration may reduce charging artifacts for one type of specimen but not another). Therefore, existing techniques cannot be easily or readily implemented across different types of charged particle microscopes and different types of specimens. This significant lack of generality of existing techniques can be considered disadvantageous.
[0018] Therefore, a system or technique that can eliminate or eliminate charging artifacts in a general way is desirable.
[0019] Various embodiments described herein may address this technical problem. One or more embodiments described herein may include a system, computer-implemented method, apparatus, or computer program product that can facilitate mitigation of charging artifacts by rotating the scan direction. In other words, the inventors of various embodiments described herein recognized that charging artifacts can depend on the direction of the scan (e.g., recognized that charging artifacts can be anisotropic) and, therefore, can be improved by rotating or otherwise manipulating the scan direction of a charged particle microscope. In particular, when a charged particle microscope scans a specimen to capture an image, charging artifacts in the image (e.g., prominent light or dark streaks) may be oriented along any of the scan directions applied by the charged particle microscope for that scan, or otherwise generally parallel to any of the scan directions. Various embodiments described herein may include obtaining multiple scan images of the specimen by scanning the specimen multiple times along different scan directions that rotate the specimen symmetrically relative to one another. Various embodiments described herein may also include aggregating or averaging these multiple scan images together (after performing corrective rotation or cropping) to obtain a single resulting image of the specimen. Because the multiple scan images have different scanning directions and the charging artifact is anisotropic, the multiple scan images can show a specimen with differently oriented charging artifacts. In other words, a portion of the specimen that is obscured or distorted by the charging artifact in some of the multiple images can be hidden or distorted in other of the multiple images. In other words, rotation of the scanning direction can cause the charging artifact to be located in different positions in different of the multiple images. Therefore, by aggregating or averaging the multiple images, the charging artifacts can be collectively eliminated or eliminated, and the resulting single image can show the specimen without (or with significantly less noticeable) charging artifacts.The reduction or elimination of such charging artifacts can be easily implemented regardless of the type of specimen and the type of microscope, and therefore the various embodiments described herein are believed to be significantly more general than existing techniques.
[0020] Various embodiments described herein can be considered as computerized tools (e.g., any suitable combination of computer-executable hardware or computer-executable software) that can facilitate charging artifact mitigation through rotation of the scan direction. In various aspects, such computerized tools can include an access component, a scan component, an alignment component, a crop component, or an aggregation component.
[0021] In various embodiments, a charged particle microscope can be present. In various aspects, the charged particle microscope can exhibit any suitable design or configuration (e.g., can be an SEM or can be a dual beam microscope). In various cases, there can be any suitable specimen (e.g., a semiconductor wafer or thin plate) mounted on the charged particle microscope (e.g., currently placed or positioned on an operable stage of the charged particle microscope). In various cases, the charged particle microscope can have selectively adjustable settings or parameters that control the scan direction of the charged particle microscope (e.g., control which axis the charged particle microscope uses to raster scan the specimen). In various cases, the charged particle microscope can have a 360° range from which a scan direction can be picked or selected.
[0022] In various embodiments, it is desirable to obtain a scan image of a specimen via a charged particle microscope and according to or aligned with a suitable target scan direction (e.g., any scan direction from a 360° range of possible scan directions). However, in various cases, the specimen may have a physical or chemical composition that is likely to produce undesirable charging artifacts (e.g., the specimen may be ceramic). As described herein, a computerized tool can generate an image of the specimen aligned with the target scan direction and with little or no charging artifacts.
[0023] In various embodiments, an access component of the computerized tool can electronically access the charged particle microscope. For example, the access component can electronically interface with or communicate with the charged particle microscope (e.g., send electronic instructions to the charged particle microscope and read electronic signals from the charged particle microscope). In some cases, the access component can be considered a conduit through which other components of the computerized tool can electronically interact with (e.g., read, write, edit, copy, manipulate, execute, start, stop) the charged particle microscope.
[0024] In various embodiments, a scanning component of the computerized tool can cause the charged particle microscope to electronically acquire multiple images of the specimen according to a target scan direction and multiple rotated scan commands.
[0025] More specifically, the scanning component can electronically instruct the charged particle microscope to scan the specimen using a target scan direction. Non-scanning parameters of the charged particle microscope (e.g., scan speed, beam voltage, focus spot size, detector settings, field of view size) can be set to appropriate values, states, or configurations during such a scan. In either case, such a scan can cause the charged particle microscope to generate or capture a target image of the specimen. In various aspects, the target image can be a pixel array or voxel array depicting the specimen (e.g., depicting surface details or internal details of the specimen). In various cases, the field of view utilized by the charged particle microscope during that scan can be referred to as the target field of view. In various cases, the physical or chemical composition of the specimen can contaminate or damage the target image with charging artifacts along the target scan direction.
[0026] Here, the multiple rotational scan directions can include any suitable number of rotational scan directions. In various aspects, the rotational scan directions can be any suitable scan directions that can be implemented in the charged particle microscope (e.g., chosen or selected from a 360° range of possible scan directions) and rotate around the center or centroid of the target field of view at respective or unique angular intervals from the target scan direction. Similarly, each rotational scan direction can be obtained by rotating the target scan direction around the center or centroid of the target field of view by respective or unique angular displacements. In some cases, the target scan direction and the multiple rotational scan directions can be set together radially symmetrically with respect to each other. In other words, the target scan direction and the multiple rotational scan directions can both cover a 360° rotation via equally spaced angular increments. In other words, the target scan direction and the multiple rotational scan directions can be considered to be uniformly distributed over the 360° range of the charged particle microscope.
[0027] As a non-limiting example, assume the plurality of rotational scan directions includes seven directions. If the target scan directions and the seven rotational scan directions are radially symmetrically or uniformly distributed, the target scan directions and the seven rotational scan directions can be evenly spaced by 45° increments. A first rotational scan direction can be positioned 45° clockwise from the target scan direction, a second rotational scan direction can be positioned 45° clockwise from the first rotational scan direction that is 90° clockwise from the target scan direction, a third rotational scan direction can be positioned 45° clockwise from the second rotational scan direction that is 135° clockwise from the target scan direction, and a fourth rotational scan direction can be positioned 180° clockwise from the target scan direction. a fifth rotational scan direction can be positioned at 45° clockwise from the fourth rotational scan direction and at 225° from the target scan direction; a sixth rotational scan direction can be positioned at 45° clockwise from the fifth rotational scan direction and at 270° from the target scan direction; and a seventh rotational scan direction can be positioned at 45° clockwise from the sixth rotational scan direction and at 315° clockwise from the target scan direction.
[0028] As another non-limiting example, assume the plurality of rotational scan directions includes 11 directions. The target scan directions and the 11 rotational directions can be evenly spaced by 30° increments if the target scan directions and the 11 rotational directions are radially symmetric or uniformly distributed. The first rotational scan direction can be positioned at 30° clockwise from the target scan direction, the second rotational scan direction can be positioned at 60° clockwise from the target scan direction, or 30° clockwise from the first rotational scan direction, the third rotational scan direction can be positioned at 90° clockwise from the target scan direction, or 30° clockwise from the second rotational scan direction, the fourth rotational scan direction can be positioned at 120° clockwise from the target scan direction, or 30° clockwise from the third rotational scan direction, the fifth rotational scan direction can be positioned at 150° clockwise from the target scan direction, or 30° clockwise from the fourth rotational scan direction, and the sixth rotational scan direction can be positioned at 180° clockwise from the fifth rotational scan direction. the seventh rotational scan direction can be positioned at a position 30° clockwise from the sixth rotational scan direction that is 210° clockwise from the target scan direction; the eighth rotational scan direction can be positioned at a position 30° clockwise from the seventh rotational scan direction that is 240° clockwise from the target scan direction; the ninth rotational scan direction can be positioned at a position 30° clockwise from the eighth rotational scan direction that is 270° clockwise from the target scan direction; the tenth rotational scan direction can be positioned at a position 30° clockwise from the ninth rotational scan direction that is 300° clockwise from the target scan direction; and the eleventh rotational scan direction can be positioned at a position 330° clockwise from the target scan direction that is 30° clockwise from the tenth rotational scan direction.
[0029] However, these are merely non-limiting examples. In another aspect, the target scan direction and the multiple rotational scan directions may not be non-radially symmetric with respect to one another. In other words, both the target scan direction and the multiple rotational scan directions may span a 360° rotation (or even less than 360° rotation) at non-equidistant angular increments. In yet another way, the target scan direction and the multiple rotational scan directions may be non-uniformly distributed across the 360° range of the charged particle microscope.
[0030] In any case, the scanning component can electronically instruct the charged particle microscope to scan the specimen using each of multiple rotational scan directions. This can generate multiple rotational images, each of which can be a pixel or voxel array showing the specimen from a unique or different rotation. In various aspects, the non-scanning direction parameter values or conditions (e.g., scan speed, beam voltage, focal spot size, detector settings) used by the charged particle microscope to generate the target image can also be used to generate each of the multiple rotational images, excluding the field of view. In particular, each of the multiple rotational scan directions can have a respective field of view that is expanded or otherwise resized to circumscribe the target field of view. Indeed, without such expansion or resizing, some of the multiple rotational images may crop or otherwise omit corners of the target field of view. Thus, due to such enlargement or resizing, the number of pixels or voxels included in each of the plurality of rotated images may be equal to or greater than the number in the target image, such that any visual content shown in the target image is shown in each of the plurality of rotated images (from a rotated perspective or orientation, but using the same spatial resolution) (e.g., no portion of the target image is cut off or omitted in any of the plurality of rotated images.) In other words, some of the plurality of rotated images may be the same size (in terms of number or arrangement of pixels or voxels) as each other or as the target image, while other of the plurality of rotated images may be different sizes (in terms of number or arrangement of pixels or voxels) from each other or as the target image.
[0031] It should be noted that, as noted above, the physical or chemical composition of the specimen may contaminate or corrupt each image of the plurality of rotated images with charging artifacts oriented along each of the plurality of rotational scan directions. Thus, for any two images selected from a superset containing the target image and the plurality of rotated images, the charging artifacts in the two selected images may obscure (e.g., may be located on or resting on) different or non-identifiable portions of the specimen.
[0032] In various embodiments, the alignment component can leave the target image unchanged. However, the alignment component can electronically edit or otherwise manipulate the multiple rotated images to align or register their visual content with that of the target image. After such editing or manipulation, the multiple rotated images can be referred to as multiple aligned images. More specifically, for each given rotated image, the alignment component can perform a corrective rotation about the center or centroid of the target field of view by an angular displacement to move the respective scan direction away from the target scan direction, thereby generating a respective aligned image. Before such corrective rotation, any portion of the specimen depicted in the rotated image can be displayed with the same spatial resolution as depicted in the target image, but from a different perspective or orientation (e.g., the rotated image may appear completely or partially upside down or sideways relative to the target image). However, after such corrective rotation, any portion of the specimen depicted in a given rotated image (referred to herein as an aligned image) can be displayed with the same spatial resolution and from the same perspective or orientation as depicted in the target image. In some cases, other suitable image registration or alignment techniques, such as optical drift correction techniques, can be implemented by the alignment component in combination with such correction rotations. In some cases, multiple aligned images can all be shown at the same spatial resolution and from the same perspective or direction as that shown in the target image.
[0033] In various embodiments, the crop component of the computerized tool can leave the target image unchanged. However, the crop component can electronically edit or manipulate the multiple aligned images so that they contain the same number or arrangement of pixels or voxels as the target image. After such editing or manipulation, the multiple aligned images can be referred to as multiple cropped images. Indeed, as noted above, the number of pixels or voxels in each of the multiple rotated images, and therefore each of the multiple aligned images, can be equal to or greater than those in the target image. In other words, various images of the multiple aligned images can have extraneous pixels or voxels that are not present in the target image (e.g., such pixels or voxels can indicate visual content beyond or outside the range of the target field of view). In various aspects, for a given aligned image, the crop component removes, deletes, or otherwise crops out any pixels or voxels from the given aligned image that are not included in the target image, thereby generating a respective cropped image. This allows each of the multiple cropped images to depict or show the specimen from the same perspective or direction as the target image using the same spatial resolution, and the cropped images can also be the same size (in terms of number or arrangement of pixels or voxels) as the target image.
[0034] Currently, in various embodiments, an aggregation component of a computerized tool can electronically generate an aggregate image based on the multiple cropped images. More specifically, each of the multiple cropped images can be the same size (in terms of the number or arrangement of pixels or voxels) as the target image, so that the aggregation component can average the target image and the multiple cropped images together (e.g., via a pixel-by-pixel or voxel-by-voxel basis). In various aspects, the result of such averaging may be referred to as an aggregate image. In various cases, the aggregate image can be the same size (in terms of the number or arrangement of pixels or voxels) as the target image. Furthermore, the aggregate image can show the specimen from the same perspective or orientation as the target image, using the same spatial resolution as the target image. However, unlike the target image, the aggregate image does not contain charging artifacts (or can contain significantly less or significantly reduced versions thereof). Indeed, as noted above, each of the multiple rotational images, and therefore each of the multiple cropped images, may have a uniquely positioned, uniquely located, or otherwise unique spatially distributed charging artifact (e.g., the charging artifact in any given cropped image may be oriented parallel to the rotational scan direction from which the given cropped image was derived). Therefore, during the averaging calculation, the various charging artifacts collectively shown in the target image and the multiple cropped images may refrain from constructively interfering with each other. In other words, each unique or individual charging artifact may be shown in only one (or otherwise a small proportion of) the superset containing the target image and the multiple cropped images, and thus each unique or individual charging artifact may be considered to contain or retain very little weight in or during the averaging calculation. Therefore, all of the various charging artifacts may be considered to have been dropped, disappeared, or otherwise significantly reduced during the averaging calculation.In contrast, various features or portions of the specimen may be depicted in the same location or position across a superset that includes the target image and multiple cropped images, and thus the features or portions of the specimen may be considered to carry or carry significant weight in or during the averaging calculation. Thus, while charging artifacts may be dropped or otherwise reduced by averaging, the features or portions of the specimen may be preserved or otherwise enhanced by averaging.
[0035] Thus, various embodiments described herein enable a charged particle microscope to acquire or generate images of a specimen according to a target scan direction, and such images are free of (or contain few, significantly less noticeable, or imperceptible) charging artifacts. It should be noted that such embodiments may be applicable even in the absence of charge neutralizing agents, conductive specimen coatings, or other niche equipment specifically designed to mitigate charging artifacts. It should also be noted that such embodiments may be applied to specimens regardless of the specimen's physical or chemical composition. Thus, various embodiments described herein may be considered very general or universal techniques for mitigating or attenuating charging artifacts (e.g., applicable across all charged particle microscopes and all specimens).
[0036] Various embodiments described herein may use hardware or software to solve problems that are highly technical in nature (e.g., to facilitate mitigation of charging artifacts through rotation of scan direction), not abstract, and not implementable as a series of mental acts by a human. Furthermore, some of the processes performed may be performed by specialized computers (e.g., electron microscopes such as SEMs and dual-beam microscopes) to perform defined acts related to the field of charged particle microscopy.
[0037] For example, such defined acts may include accessing, by a device operatively coupled to the processor, a charged particle microscope carrying a specimen, and generating, by the device, an aggregate image of the specimen based on multiple images of the specimen captured by the charged particle microscope according to a target scan direction and multiple rotational scan directions. In various embodiments, the multiple rotational scan directions and the target scan directions may be uniformly distributed within a 360° range. In various cases, a target image of the multiple images may be captured according to the target scan direction and may have a target field of view of the specimen, and the remaining images of the multiple images may each have a resized field of view circumscribing the target field of view. In various cases, such defined acts may include: correctively rotating, by the device, each of the plurality of images according to a plurality of rotational scan directions to thereby generate a plurality of aligned images aligned with the target field of view; applying, by the device, a drift correction to each of the plurality of aligned images; cropping, by the device and outside the plurality of aligned images, each of any pixels or voxels not present in the target field of view to thereby generate a plurality of cropped images each having the same size as the target image; and averaging, by the device, the plurality of cropped images and the target image together to generate an aggregate image. In various aspects, the specimen may be non-uniformly charged during scanning, and each of the plurality of images may exhibit a charging artifact, and the aggregate image may exhibit no or visibly reduced charging artifact.
[0038] Such defined actions are inherently computerized. Indeed, charged particle microscopes (e.g., SEMs, dual-beam microscopes) are highly computerized devices that include specific computerized hardware (e.g., temperature sensors, pressure sensors, voltage sensors, ion beam emitters, electron beam emitters, focusing lenses, ion detectors, electron detectors, beam apertures, fluid valves, and activatable specimen stages). Charged particle microscopes and the operations performed thereby cannot be performed by a human mind or a person with pen and paper in any reasonable or practical manner other than by a computer. Furthermore, all image editing or manipulation (e.g., rectifying, rotating, registering, cropping) described herein are inherently computerized actions performed on scanned images produced by a charged particle microscope. Neither a human mind nor a person with merely pen and paper can rotate or crop pixel or voxel arrays produced by a charged particle microscope. Furthermore, mitigating or reducing charging artifacts is an inherently computerized task. Neither the human mind nor a person simply holding a pen and paper can mitigate or reduce the visibility of charging artifacts depicted in pixel or voxel arrays captured by charged particle microscopes.
[0039] Furthermore, various embodiments described herein can integrate various teachings related to the field of charged particle microscopy into practical applications. As described above, charged particle microscopes can generate or capture scanned images of specimens, some of which can be unevenly charged, causing undesirable charging artifacts (e.g., streaks, shadows, and other distortions) that adversely affect the resulting scanned images. Some existing techniques attempt to address charging artifacts by using niche or rarely encountered microscope hardware (e.g., charge neutralizers, conductive specimen coaters). Unfortunately, most charged particle microscopes do not have or incorporate such niche or rarely encountered hardware and therefore cannot benefit from such existing techniques. Other existing techniques attempt to address charging artifacts by tuning operating parameters (e.g., scan speed, beam voltage, detector settings) to specific values or conditions that are experimentally known to minimize charging artifacts for any given specimen. Unfortunately, such other existing techniques require significant experimentation and tinkering for each different specimen type and composition. Thus, existing techniques can be considered as not easily generalizable or universal across different charged particle microscopes and different types of specimens.
[0040] Various embodiments described herein can help ameliorate this problem by implementing charging artifact mitigation through scan direction rotation. Rather than capturing only one image of a given specimen, various embodiments described herein can instead capture multiple images of the specimen, each according to a corresponding number of rotational scan directions. Indeed, the charging artifact can be anisotropic and thus physically oriented along or parallel to the scan direction used. Thus, each of the multiple images can depict the specimen according to a distinct or unique scan direction and, accordingly, can have distinct or unique charging artifacts that are not co-located with charging artifacts in other images of the multiple images. In various aspects, various embodiments described herein can edit the multiple images (e.g., via corrective rotation or cropping) so that the images are the same size and depict the specimen from the same perspective or orientation. In various cases, various embodiments described herein can average the multiple images together after such editing, thereby generating an aggregate image. Unlike multiple images, the aggregate image can depict the specimen without charging artifacts (or with inconspicuous or imperceptible charging artifacts). Indeed, because each of the multiple images can be considered to have a charging artifact in its unique location or unique orientation, the act of averaging the multiple images together can cause all charging artifacts to be dropped or nearly eliminated (e.g., each unique or individual charging artifact can be present in only a small percentage of the multiple images, allowing each unique or individual charging artifact to be considered to have low or nearly no weight during averaging). In this way, charging artifacts can be reduced or eliminated regardless of what type of charged particle microscope is implemented and what type of specimen is scanned.In other words, the various embodiments described herein can be considered distinct or innovative techniques for reducing, mitigating, or attenuating charging artifacts that are applicable or generalizable across a variety of charged particle microscopes (unlike existing techniques that rely on niche microscope hardware) and across different specimens (unlike existing techniques that rely on specimen-specific parameter optimization). For at least the above reasons, the various embodiments described herein can be considered to address or ameliorate various problems and deficiencies that adversely affect existing techniques for mitigating charging artifacts. Accordingly, the various embodiments described herein can be considered concrete and tangible technical improvements in the field of charged particle microscopy. As such, the various embodiments described herein certainly qualify as useful and practical applications of computers.
[0041] Additionally, various embodiments described herein can control real-world, tangible devices based on the disclosed teachings. For example, various embodiments described herein can electronically activate, deactivate, or otherwise actuate real-world hardware (e.g., ion beam emitters, ion focusing lenses, carrier fluid valves / pumps) of a real-world charged particle microscope (e.g., SEM, dual beam microscope).
[0042] FIG. 1 illustrates an example, non-limiting block diagram of a scientific instrument module 102 according to various embodiments described herein.
[0043] In various embodiments, the scientific instrument module 102 may be implemented by circuitry (e.g., including electrical and / or optical components) such as a programmed computing device. The logic of the scientific instrument module 102 may be contained in a single computing device or distributed across multiple computing devices that communicate with each other as needed. Examples of computing devices that may implement the scientific instrument module 102, alone or in combination, are discussed herein with reference to Figures 22 and 24, and examples of systems or networks of interconnected computing devices in which the scientific instrument module 102 may be implemented across one or more of the computing devices are discussed herein with reference to Figures 23 and 25.
[0044] The scientific instrument module 102 may include first logic 104 and second logic 106. As used herein, the term “logic” may include a device that performs a set of operations associated with the logic. For example, any of the logic elements included in the scientific instrument module 102 may be implemented by one or more computing devices programmed with instructions that cause one or more processing units of the computing devices to perform an associated set of operations. In particular embodiments, a logic element may include one or more non-transitory computer-readable media having instructions thereon that, when executed by one or more processing units of the one or more computing devices, cause the one or more computing devices to perform the associated set of operations. As used herein, the term “module” may refer to a collection of one or more logic elements that together perform the functionality associated with the module. Different ones of the logic elements within a module may take the same form or different forms. For example, some logic within a module may be implemented by a programmed general-purpose processing unit, while other logic within the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different logic elements within a module may be associated with different sets of instructions executed by one or more processing units. A module may omit one or more of the logic elements shown in the associated drawings. For example, a module may include a subset of the logic elements shown in the associated drawings when the module performs a subset of the operations discussed herein with reference to the module.
[0045] In various embodiments, there can be a scientific instrument corresponding to the scientific instrument module 102. In various aspects, the scientific instrument can be any suitable computerized device capable of electronically measuring some scientifically, clinically, or research-related characteristic, property, or attribute of an analytical specimen (e.g., a known or unknown mixture, compound, or collection of substances). As a non-limiting example, the scientific instrument can be a scanning electron microscope. In such a case, the scientific instrument can capture images of the analytical specimen to measure or determine the surface topography, surface material composition, or crystallographic structure of the analytical specimen. As another non-limiting example, the scientific instrument can be a dual-beam microscope. In such a case, the scientific instrument can capture images of the analytical specimen in addition to being able to crush the analytical specimen. As a further general non-limiting example, the scientific instrument can be a suitable type of charged particle microscope (e.g., some types of microscopes can capture images using light of non-electronic ions).
[0046] In various embodiments, the first logic 104 can access the charged particle microscope to read electronic data from the charged particle microscope and send electronic commands to the charged particle microscope. In various cases, the charged particle microscope can be loaded with a suitable specimen.
[0047] In various embodiments, the second logic 106 may generate an aggregate image of the specimen based on multiple images of the specimen captured by the charged particle microscope according to a target scan direction and multiple rotational scan directions. More specifically, each of the multiple rotational scan directions may have several unique or respective angular displacements from the target scan direction. In various aspects, the second logic 106 may instruct the charged particle microscope to scan the specimen according to the target scan direction, thereby generating the target image. Further, the second logic 106 may instruct the charged particle microscope to scan the specimen according to each of the multiple rotational scan directions, thereby generating the multiple rotated images. In various cases, the second logic 106 may correctively rotate and align or register the multiple rotated images to the target image, thereby generating the multiple aligned images. In various cases, the second logic 106 may crop pixels or voxels from each of the multiple aligned images that are not present in the target image, thereby generating the multiple cropped images. In various embodiments, the second logic 106 can include averaging the target image and the multiple cropped images together, thereby generating an aggregate image. In various cases, the aggregate image can depict the specimen from the same perspective or orientation as the target image. However, the aggregate image can include fewer or less noticeable charging artifacts compared to the target image and compared to each of the multiple cropped images.
[0048] In this manner, the scientific instrument module 102 can facilitate mitigation of charging artifacts through rotation of the scan direction, regardless of the type of charged particle microscope and regardless of the physical or chemical composition of the specimen.
[0049] 2 is a non-limiting example flow diagram of a computer-implemented method 200 according to various embodiments described herein. The operations of computer-implemented method 200 may be used in any suitable context to perform any suitable operations (e.g., performed by or used in conjunction with any of the various modules, computing devices, or graphical user interfaces described with respect to FIGS. 1, 21, 22, 23, 24, and 25). Although the operations are described in FIG. 2 once each and in a particular order, the operations may be appropriately reordered or repeated as desired (e.g., different operations performed may be performed in parallel, as appropriate).
[0050] In various aspects, act 202 can include performing a first operation of accessing a charged particle microscope with a specimen mounted thereon. In various cases, first logic 104 can perform or otherwise facilitate act 202.
[0051] In various embodiments, act 204 may include performing a second operation of generating an aggregate image of the specimen based on a plurality of images of the specimen captured by the charged particle microscope according to the target scan direction and the plurality of rotational scan directions. In various cases, second logic 106 may include performing or otherwise facilitating act 204.
[0052] In this manner, the computer-implemented method 200 can facilitate mitigation of charging artifacts through rotation of the scan direction.
[0053] FIG. 3 illustrates a block diagram of a non-limiting example system that can facilitate charging artifact mitigation through rotation of scan direction, according to one or more embodiments described herein.
[0054] In various embodiments, there can be a charged particle microscope 302. In various aspects, the charged particle microscope 302 can be as described above. That is, the charged particle microscope 302 can be any suitable computerized device capable of electronically capturing any suitable image of any suitable analytical specimen utilizing its constituent hardware (e.g., electron source, anode, condenser lens, condenser aperture, scan coil, objective lens, objective aperture, deflector, condenser, astigmatism corrector, electron detector, X-ray detector, movable specimen stage). As a non-limiting example, the charged particle microscope 302 can be any suitable SEM. As another non-limiting example, the charged particle microscope 302 can be any suitable dual beam microscope.
[0055] Although not explicitly shown in the figures, the charged particle microscope 302 may be electronically integrated with any suitable human-computer interface device, which may be remote from the charged particle microscope 302 or local to the charged particle microscope 302. In this manner, a user or technician associated with the charged particle microscope 302 may interact with or otherwise control the charged particle microscope 302. Some non-limiting examples of human-computer interface devices may be a keyboard of the charged particle microscope 302, a keypad of the charged particle microscope 302, a touchscreen of the charged particle microscope 302, or a voice command system of the charged particle microscope 302.
[0056] Although not explicitly shown in the figures, the charged particle microscope 302 can include or otherwise be associated with any suitable number of configurable operating parameters. In various aspects, the configurable operating parameters can be any suitable hardware- or software-related settings of the charged particle microscope 302 that can guide, influence, or otherwise dictate how the charged particle microscope 302 operates, operates, or functions for any given analytical analyte, and that can be selectively controlled, changed, adjusted, or otherwise set by a user or technician (e.g., via interaction with a human-computer interface device of the charged particle microscope 302). As a non-limiting example, a configurable operating parameter can be a user-controllable voltage setting (e.g., beam voltage) or current setting (e.g., beam current) that enables a user or technician to selectively control electrodes of the charged particle microscope 302 to selectively increase or decrease the voltage or current applied in or by the charged particle microscope 302. As another non-limiting example, a configurable operating parameter can be a user-controllable temperature setting that allows a user or technician to control a heater (e.g., stage heater, heating coil) or cooler (e.g., cooling fan, heat pump, refrigerator) of the charged particle microscope 302 to selectively increase or decrease the temperature within or applied by the charged particle microscope 302. As yet another non-limiting example, a configurable operating parameter can be a user-controllable mechanical actuator setting that allows a user or technician to control a mechanical actuator (e.g., an electric motor, specimen stage, aperture, fluid pump, or syringe) of the charged particle microscope 302 to selectively move the mechanical actuator.As yet another non-limiting example, a configurable operating parameter may be a user-controllable optical setting that allows a user or technician to control optical elements (e.g., optical lenses, optical deflectors) of the charged particle microscope 302 to selectively change the optical qualities (e.g., focal spot size or position, astigmatism, defocus) applied by the charged particle microscope 302. In various cases, any configurable operating parameter may be alterable or adjustable remotely or automatically.
[0057] In various embodiments, one of the configurable operating parameters of the charged particle microscope 302 can be a scan direction setting. In various cases, the scan direction setting can control or manage the scan direction performed by the charged particle microscope 302 during any given scan. In various cases, the scan direction can refer to or be the axis along which the charged particle microscope 302 raster scans (e.g., back and forth or in a zigzag manner) the surface or interior of any given specimen. In various embodiments, the charged particle microscope 302 can utilize or implement any suitable scan direction from a 360° rotation range. As a non-limiting example, there can be any suitable reference plane parallel to the specimen stage of the charged particle microscope 302, which can include a center and a reference axis passing through the center. In such a case, there can be another axis in the reference plane, which passes through the center and is angularly offset from the reference axis by φ degrees, for any suitable real number 0≦φ≦360. In this manner, such other axis can be implemented or utilized as a scan direction by the charged particle microscope 302. In some aspects, the scan direction setting can be considered as selectively or controllably configuring φ.
[0058] In various cases, another configurable operating parameter of the charged particle microscope 302 can be a field of view (FOV) setting. In various cases, the FOV setting can control or manage the FOV by the charged particle microscope 302 during any given scan. In various embodiments, the FOV can refer to or be a linear window of length L and width W, where L and W can be any suitable positive real numbers and define the total area that the charged particle microscope 302 raster scans during any given scan. In other words, any portion of any given specimen or specimen stage that falls within the FOV can be considered to be observed, viewed, or imaged by the charged particle microscope 302, while a portion of a given specimen or specimen stage that is outside the FOV can be considered not to be observed, viewed, or imaged by the charged particle microscope 302. In some cases, L and W can range from single-digit nanometers to thousands of micrometers. In some embodiments, the FOV setting can be considered to selectively or controllably configure L and W.
[0059] In various cases, the charged particle microscope 302 can be loaded with a specimen 304. As a non-limiting example, the specimen 304 can be presently positioned, placed, or otherwise attached to a specimen stage of the charged particle microscope 302 such that the specimen 304 can be analyzed or scanned by the charged particle microscope 302. In various cases, the specimen 304 can be any suitable type of medical, clinical, scientific, or laboratory specimen that can exhibit any suitable physical, chemical, compositional, or other property, characteristic, or feature. As a non-limiting example, the specimen 304 can be a thin slab taken from a semiconductor substrate or wafer. As another non-limiting example, the specimen 304 can be a sample or fragment taken from a failed load-bearing structure. As yet another non-limiting example, the specimen 304 can be a biological or organic sample.
[0060] In various cases, there can be a target scan direction 322. In various aspects, the target scan direction 322 can be any suitable scan direction implementable or usable by the charged particle microscope 302. In various cases, it is desirable for the charged particle microscope 302 to generate an image of the specimen 304 using or otherwise following the target scan direction 322. However, it is desirable for such an image to be free of charging artifacts or have the least visually noticeable charging artifacts. As described herein, the system 306 can facilitate or achieve such an image.
[0061] In various embodiments, the system 306 may be electronically integrated with the charged particle microscope 302 or the target scan direction 322 via any suitable wired or wireless electronic connection.
[0062] In various aspects, the system 306 may include a processor 308 (e.g., a computer processing unit, microprocessor) and non-transitory computer-readable memory 310 functionally, operatively, or communicatively connected or coupled to the processor 308. The non-transitory computer-readable memory 310 may store computer-executable instructions that, when executed by the processor 308, cause the processor 308 or other components of the system 306 (e.g., the access component 312, the scan component 314, the alignment component 316, the crop component 318, the aggregation component 320) to perform one or more actions. In various embodiments, the non-transitory computer-readable memory 310 may store, and the processor 308 may execute, the computer-executable components (e.g., the access component 312, the scan component 314, the alignment component 316, the crop component 318, the aggregation component 320).
[0063] In various embodiments, the system 306 can include an access component 312. In various aspects, the access component 312 has electronic access to the charged particle microscope 302. That is, the access component 312 can electronically communicate with or otherwise interact with the charged particle microscope 302 (e.g., send electronic commands or instructions, receive electronic data, etc.). In this manner, the access component 312 can be considered a proxy or conduit through which other components of the system 306 can interact, communicate with, or otherwise manipulate the charged particle microscope 302. In various cases, the access component 312 has electronic access to the target scan direction 322. That is, the access component 312 can electronically receive, retrieve, or otherwise obtain the target scan direction 322 from any suitable electronic source or database (e.g., from the charged particle microscope 302 or an associated computerized workstation, as the case may be). (The target scan direction 322 can be indicated by user input to the charged particle microscope 302 or an associated computerized workstation.) In either case, the access component 312 can be considered a proxy or conduit through which other components of the system 306 can interact, control, or otherwise manipulate the target scan direction 322. However, these are merely non-limiting examples. In other cases, the access component 312 can be omitted, and any other component of the system 306 can communicate or interact directly with the charged particle microscope 302 or the target scan direction 322.
[0064] In various embodiments, the system 306 can include a scanning component 314. In various aspects, the scanning component 314 can cause the charged particle microscope 302 to capture multiple images of the specimen 304 according to a target scan direction 322 and according to multiple other scan instructions, each rotated from the target scan direction 322, as described herein.
[0065] In various embodiments, the system 306 can include an alignment component 316. In various cases, the alignment component 316 can register or align multiple images to the captured image according to a target scan direction, as described herein, thereby generating multiple aligned images.
[0066] In various embodiments, the system 306 can include a crop component 318. In various cases, the crop component 318 can crop out from each of the multiple aligned images any pixels or voxels that are not located in any of the captured images according to the target scan direction, as described herein, thereby generating multiple cropped images.
[0067] In various embodiments, the system 306 can include an aggregation component 320. In various aspects, the aggregation component 320 can aggregate multiple cropped images together, thereby generating a single aggregate image, as described herein. In various cases, the aggregate image can depict the specimen 304 according to the target scan direction 322, while the aggregate image can lack (or have significantly reduced) charging artifacts.
[0068] It is noted that in various cases, the access component 312, the scan component 314, the alignment component 316, the crop component 318, and the aggregation component 320 can be considered as one or more software components 311 of the system 306. It should be understood that in various aspects, for ease of explanation and illustration, the one or more software components 311 are primarily described herein as including five components (e.g., the access component 312, the scan component 314, the alignment component 316, the crop component 318, and the aggregation component 320). However, the one or more software components 311 are not limited to being implemented as exactly five such components in all embodiments. Indeed, in some embodiments, the functionality described herein for such five components can be combined in any suitable manner to be implemented with five components or with fewer than five components (e.g., in some cases, one component can perform all of the functionality described herein for the access component 312, the scan component 314, the alignment component 316, the crop component 318, and the aggregation component 320). In other embodiments, the functions described herein for such five components may instead be distributed, separated, divided, or fragmented in any suitable manner so as to be implemented in five components or in more than four components (e.g., two or more components may facilitate functions performable in access component 312; two or more components may facilitate functions performable in scan component 314; two or more components may facilitate functions performable in align component 316; two or more components may facilitate functions performable in crop component 318; two or more components may facilitate functions performable in aggregate component 320).
[0069] FIG. 4 illustrates a block diagram of a non-limiting example system including multiple rotational scan directions and multiple captured images that facilitates charging artifact mitigation through scan direction rotation, according to one or more embodiments described herein.
[0070] In various embodiments, the scan component 314 can electronically generate multiple captured images 404 of the specimen 304 based on the target scan direction 322 and based on multiple rotational scan commands 402. Non-limiting aspects are described with respect to FIGS.
[0071] 5-8 show non-limiting example flow diagrams illustrating methods for implementing multiple rotational scan directions 402 to acquire multiple images 404 according to one or more embodiments described herein.
[0072] Referring first to FIG. 5 , in various embodiments, the scan component 314 can electronically command, instruct, or otherwise cause the charged particle microscope 302 to capture or generate an image of the specimen 304 using or according to the target scan direction 322. In various aspects, the image obtained from such a scan or the image acquired by such a scan can be referred to as a target image 502. In various cases, the target image 502 can exhibit any suitable format, size, or dimensions. As a non-limiting example, the target image 502 can be an x-by-y array of pixels, where x and y are any suitable positive integers. As another non-limiting example, the target image 502 can be an x-by-y-by-z array of pixels, where x, y, and z are any suitable positive integers. For ease of explanation and illustration, the target image 502 disclosed herein (and various other images described herein) is treated as a two-dimensional pixel array, although it should be understood that the teachings described herein are also applicable to three-dimensional voxel arrays.
[0073] In various cases, the charged particle microscope 302 can utilize or implement any suitable configurable operating parameter values or states to capture or generate the target image 502. As some non-limiting examples, the charged particle microscope 302 can utilize any suitable scan speed, any suitable beam voltage, any suitable focal spot size, any suitable detector setting, or any suitable FOV to capture or generate the target image 502. In various cases, the FOV utilized or implemented by the charged particle microscope 302 to generate the target image 502 can be referred to as a target field of view 504 (hereinafter, “target FOV 504”). Although not explicitly shown in FIG. 5 , it should be understood and appreciated that the target FOV 504 can include a center or centroid.
[0074] Here, in various embodiments, the plurality of rotational scan directions 402 includes n rotational scan directions 402(1) through 402(n), where n is any suitable positive integer greater than 1. In various cases, each of the plurality of rotational scan directions 402 can be a rotational scan direction about the center or center of gravity of the target FOV 504 with a respective angular displacement away from the target scan direction 322. In various cases, each of the plurality of rotational scan directions 402 can have a unique or individual angular displacement relative to the target scan direction 322. In other words, no two of the plurality of rotational scan directions 402 can be identical to each other or to the target scan direction 322. As a non-limiting example, the rotational scan direction 402(1) can have a first angular displacement relative to the target scan direction 322, where the first angular displacement is not equal to the angular displacement of any other one of the plurality of rotational scan directions 402. As a non-limiting example, the rotational scan direction 402(n) can have an nth angular displacement relative to the target scan direction 322, where the nth angular displacement is not equal to the angular displacement of any other one of the multiple rotational scan directions 402.
[0075] In some cases, the target scan direction 322 and the plurality of rotational scan directions 402 may be radially symmetric with respect to one another. That is, the target scan direction 322 and the plurality of rotational scan directions 402 may be rotated 360° together in any suitable equally spaced or uniformly distributed manner or manner. In other words, the target scan direction 322 and the plurality of rotational scan directions 402 may be considered together to form a superset of scan directions, and any two adjacent scan directions within the superset may be
number
[0076] In either case, the scanning component 314 can electronically command, instruct, or otherwise cause the charged particle microscope 302 to capture or generate a respective image of the specimen 304 using or according to each of the plurality of rotational scan directions 402. In various cases, the images resulting from or acquired by such scans can be referred to as a plurality of rotational images 506. As a non-limiting example, the scanning component 314 can command, instruct, or cause the charged particle microscope 302 to scan the specimen 304 according to rotational scan direction 402(1), and the image acquired from such a scan can be referred to as rotational image 506(1). Similarly, as another non-limiting example, the scanning component 314 can command, instruct, or cause the charged particle microscope 302 to scan the specimen 304 according to rotational scan direction 402(n), and the image acquired from such a scan can be referred to as rotational image 506(n). In various cases, rotated image 506(1) through rotated image 506(n) may be collectively considered to form a plurality of rotated images 506.
[0077] In various cases, the target image 502 and the multiple or rotated images 506 may be collectively considered as the multiple captured images 404 .
[0078] In various aspects, the charged particle microscope 302 can capture or generate the multiple rotated images 506 by utilizing or implementing the same configurable operating parameter values or states, except for the scan direction (as will become apparent from the discussion herein) and further except for the FOV, that the charged particle microscope 302 uses to capture or generate the target image 502. In particular, the scan component 314 can instruct, command, or cause the charged particle microscope 302 to capture or generate each of the multiple rotated images 506 using a respective resized or enlarged FOV that circumscribes the target FOV 504. In this manner, each of the multiple rotated images 506 can have or exhibit a total number of pixels that is equal to or greater than the total number of pixels comprising the target image 502. As a non-limiting example, the charged particle microscope 302 can capture or generate the rotated image 506(1) using a resized FOV 508(1) oriented along the rotational scan direction 402(1) and large enough to fit the target FOV 504 therein. In some cases, the size of the resized FOV 508 can depend geometrically or trigonometrically on the angular displacement between the rotational scan direction 402(1) and the target scan direction 322. As another non-limiting example, the charged particle microscope 302 can capture or generate the rotated image 506(n) using a resized FOV 508(n) oriented along the rotational scan direction 402(n) and large enough to fit the target FOV 504 within it. As described above, the size of the resized FOV 508 can depend geometrically or trigonometrically on the angular displacement between the rotational scan direction 402(n) and the target scan direction 322. The resized FOVs 508(n) from the resized FOV 508(1) can be considered to collectively form multiple resized FOVs 508. In either case, the portion or feature of the specimen 304 shown in the target image 502 can also be shown in each of the multiple rotated images 506 (albeit from different or separate perspectives or directions).
[0079] For non-limiting clarity, reference is made to Figures 6-8.
[0080] FIG. 6 illustrates a non-limiting example of a block diagram representation of a target image 502 and a target scan direction 322. As shown, the target image 502 can visually depict at least a portion of the specimen 304. While FIG. 6 illustrates the target image 502, and thus the target FOV 504, as completely containing or encompassing the specimen 304, this is merely a non-limiting example for ease of explanation and illustration. It should be understood that the target image 502, and thus the target FOV 504, can instead contain or encompass less than the entire specimen 304. In the non-limiting example of FIG. 6, the target image 502, and thus the target FOV 504, is a rectangle with a side length A, where A is any suitable positive real number. In various embodiments, A can be measured in millimeters, micrometers, or nanometers. It should be understood and appreciated that the target image 502, and therefore the target FOV 504, can be any other suitable shape (e.g., a rectangle other than a square, any other suitable rectilinear shape, and in some cases a shape other than a rectilinear shape). In various cases, the target image 502, and therefore the target FOV 504, can have a center or centroid 602. While FIG. 6 shows the target scan direction 322 pointing upward from the center or centroid 602, this is merely a non-limiting example. It should be understood and appreciated that the target scan direction 322 can pass through the center or centroid 602 and point elsewhere in the plane of FIG. 6.
[0081] Referring now to FIG. 7 , in various embodiments, there can be a rotational scan direction 702. In various aspects, the rotational scan direction 702 can be any of the multiple rotational scan directions 402. In various cases, the rotational scan direction 702 can be angularly displaced by θ degrees from the target scan direction 322 and about the center or centroid 602, where 0<θ<360 is a suitable positive real number. Note that this strict inequality can ensure that the rotational scan direction 702 is not identical to the target scan direction 322. In various cases, there can be a resized FOV 704. In various aspects, the resized FOV 704 can be any of the multiple resized FOVs 508 corresponding to the rotational scan direction 702. In particular, given a rotational scan direction 702, the resized FOV 704 may be computed or calculated via any suitable geometric or trigonometric relationship, formula, or equation such that the resized FOV 704 circumscribes the target FOV 504 and, therefore, the target image 502. As a non-limiting example, the resized FOV 704 may be a rectangle centered at the center or centroid 602 of side length B, where B=A(sin(θ)+cos(θ)).
[0082] Referring now to FIG. 8 , in various aspects, the charged particle microscope 302 can scan the specimen 304 according to a rotational scan direction 702 and according to a resized FOV 704. In various cases, such scanning can generate a rotated image 802, which can be any one of a plurality of rotated images 506 corresponding to the rotational scan direction 702 and the resized FOV 704. As shown, because the rotational scan direction 702 can be angularly displaced by θ from the target scan direction 322, the rotated image 802 can show or illustrate the specimen 304 from a different perspective or orientation than the target image 502. Indeed, if the rotational scan direction 702 is angularly displaced clockwise from the target scan direction 322, the specimen 304 can be represented in the rotated image 802 as being rotated counterclockwise by θ°. Note that rotated image 802 can be considered to show or illustrate the same substantive or visual content as target image 502 (albeit from a different or unique orientation), in addition to other substantive or visual content that is outside target image FOV 504 but within resized FOV 704.
[0083] In either case, the scanning component 314 can cause the charged particle microscope 302 to generate multiple captured images 404, which can include a target image 502 and multiple rotated images 506. In various embodiments, the charged particle microscope 302 can generate the target image 502 and each of the multiple rotated images 506 using the same, uniform, or constant focal spot size. In this manner, each of the multiple rotated images 506 can have the same spatial resolution as each other and as the target image 502. In other words, while each of the multiple rotated images 506 can have the same or larger number of pixels as the target image 502 (e.g., because their FOVs can each be resized as needed to circumscribe the target FOV 504), all pixels in the multiple captured images 404 can be considered to have or represent the same physical size as each other (e.g., all pixels can represent a 5 nanometer by 5 nanometer portion of the specimen 304).
[0084] FIG. 9 illustrates a block diagram of a non-limiting example system including multiple first edited images that can facilitate charging artifact mitigation through rotation of the scan direction, according to one or more embodiments described herein.
[0085] In various embodiments, the alignment component 316 can electronically generate the plurality of first edited images 902 based on the plurality of captured images 404. In particular, the alignment component 316 can facilitate such generation by applying any suitable corrective rotation or other image registration technique to the plurality of captured images 404. Non-limiting aspects are described with reference to FIGS.
[0086] 10-12 show non-limiting example block diagrams illustrating methods for obtaining a plurality of first edited images 902 according to one or more embodiments described herein.
[0087] 10 , in various embodiments, the alignment component 316 can electronically apply respective corrective rotations (which may be supplemented by any other suitable registration techniques) to the plurality of captured images 404 so as to align the visual content of each of the plurality of captured images 404 with the content of the target image 502. After applying such corrective rotations, the plurality of captured images 404 can be referred to as a plurality of first edited images 902.
[0088] More specifically, the alignment component 316 can save, maintain, or otherwise refrain from modifying or altering the target image 502. In this manner, the plurality of first edited images 902 can include the target image 502 as is.
[0089] Currently, in various aspects, alignment component 316 can apply a respective corrective rotation to each of the plurality of rotated images 506 to transform the plurality of rotated images 506 into the plurality of aligned images 1002 .
[0090] As a non-limiting example, alignment component 316 can rectify and rotate rotated image 506(1) by an angular displacement that separates rotated scan direction 402(1) from target scan direction 322, such that portions or features of object 304 depicted or illustrated in rotated image 506(1) are aligned with the object depicted in target image 502. In various cases, alignment component 316 can supplement such rectification and rotation using any other suitable image registration technique, such as correlation-based image registration, mutual information-based image registration, or point-based or region-based image registration (e.g., a random sample consensus (RANSAC) registration algorithm). In some cases, such supplemental registration techniques include calculating a drift or shift vector between target image 502 and the rectified, rotated version of rotated image 506(1), and using that vector to shift the rectified, rotated version of rotated image 506(1) back to target image 502. In either case, applying such corrective rotation or registration can be viewed as transforming rotated image 506(1) into aligned image 1002(1), which can have the same visual or material content as rotated image 506(1), but whose visual or material content can now be aligned with (e.g., presented from the same perspective or orientation as) the visual or material content of target image 502.
[0091] As another non-limiting example, the alignment component 316 may rectify the rotation of the rotated image 506(n) by an angular displacement that separates the rotational scan direction 402(n) from the target scan direction 322 so that portions or features of the specimen 304 shown or depicted in the rotated image 506(n) are aligned with the specimen shown in the target image 502. As noted above, the alignment component 316 may supplement such rectification rotation with any other suitable image registration technique (e.g., drift or shift correction) as desired or appropriate. In either case, the application of such rectification rotation or registration may be viewed as transforming the rotated image 506(n) into an aligned image 1002(n), which may have the same visual or material content as the rotated image 506(n), but whose visual or material content may now be aligned with (e.g., shown from the same perspective or orientation as) the visual or material content of the target image 502.
[0092] It should be noted that aligned images 1002(1) through 1002(n) can be collectively considered to form the plurality of aligned images 1002. In various instances, the plurality of aligned images 1002 and target image 502 can be collectively considered to form the plurality of first edited images 902.
[0093] For non-limiting clarification, reference is made to Figures 11-12.
[0094] FIG. 11 illustrates the rotated image 802 described above. In various embodiments, as shown, the rotated image 802 can be rectified rotated (or registered, such as via drift correction) so that its visual or material content aligns with the visual or material content of the target image 502. In the non-limiting example of FIG. 11, such rectified rotation can be performed by rotating the rotated image 802 clockwise by θ. The result of such rectified rotation is illustrated in FIG. 12. As can be seen, such rectified rotation can result in or transform the rotated image 802 into an aligned image 1202 (e.g., one of 1002) that now depicts the specimen 304 from the same perspective or orientation as the target image 502 (e.g., compare the perspective or orientation of the specimen 304 illustrated in FIG. 12 with the perspective or orientation of the specimen 304 illustrated in FIGS. 6-7).
[0095] In some cases, the corrective rotation (and other registration, if applicable) can be thought of as redefining or reorienting the pixel columns and pixel rows of rotated image 802. In fact, rotated image 802 can have pixel rows and columns aligned along rotated scan direction 702 rather than along target scan direction 322. However, aligned image 1202 can include pixel rows and columns aligned along target scan direction 322 rather than along rotated scan direction 702.
[0096] In either case, each of the plurality of first edited images 902 may show, illustrate, or depict the specimen 304 from the perspective or orientation of the target image 502.
[0097] FIG. 13 shows a block diagram of a non-limiting example system including multiple second edited images that can facilitate charging artifact mitigation through rotation of the scan direction, according to one or more embodiments described herein.
[0098] In various embodiments, the crop component 318 can electronically generate the plurality of second edited images 1302 based on the plurality of first edited images 902. In particular, the crop component 318 can facilitate such generation by applying any suitable cropping or other pixel removal technique to the plurality of first edited images 902. Non-limiting aspects are described with reference to FIGS.
[0099] 14-16 show example non-limiting block diagrams illustrating a method for obtaining the second edited image 1302 according to one or more embodiments described herein.
[0100] 14. In various embodiments, the crop component 318 may electronically crop or otherwise remove any pixels from the plurality of first edited images 902 that are not present in the target image 502 so as to align the visual content of each of the plurality of first edited images 902 with the content of the target image 502. The plurality of first edited images 902 may be referred to as a plurality of second edited images 1302 after applying such cropping or removal.
[0101] More specifically, the crop component 318 can preserve, maintain, or otherwise refrain from modifying or altering the target image 502. In this manner, the plurality of second edited images 1302 can include the target image 502 as is.
[0102] Currently, in various aspects, the crop component 318 can crop or remove any pixels that are not in the target image 502 from each of the multiple aligned images 1002 to convert the multiple aligned images 1002 into multiple cropped images 1402.
[0103] As a non-limiting example, crop component 318 can remove, delete, discard, or otherwise crop out from aligned image 1002(1) any pixels that are not present in target image 502. Indeed, aligned image 1002(1) can be considered to have or exhibit a resized FOV 508(1) that can circumscribe target FOV 504. Thus, because aligned image 1002(1) and target image 502 can have the same spatial resolution as one another, aligned image 1002(1) can be considered to include or contain all of the same pixels as target image 502 (e.g., all of the pixels that fall within target FOV 504) and additional pixels that are not in target image 502 (e.g., all of the pixels that fall within resized FOV 508(1) but that are outside of target FOV 504). Thus, crop component 318 can remove or delete pixels of aligned image 1002(1) that are not in target image 502, and such removal or deletion can result in cropped image 1402(1). In other words, cropped image 1402(1) can be the remaining image of aligned image 1002(1) after removing any pixels from aligned image 1002(1) that are not in target image 502. In this manner, cropped image 1402(1) can be considered to be the same size as (e.g., have the same number or arrangement of pixels as) target image 502.
[0104] As another non-limiting example, the crop component 318 may remove, delete, discard, or otherwise crop out from the aligned image 1002(n) any pixels that are not present in the target image 502. Indeed, the aligned image 1002(n) may be considered to have or exhibit a resized FOV 508(n) that may circumscribe the target FOV 504. Thus, because the aligned image 1002(n) and the target image 502 may have the same spatial resolution as one another, the aligned image 1002(n) may be considered to include or contain all of the same pixels as the target image 502 (e.g., all of the pixels that fall within the target FOV 504) and may also include or contain additional pixels that are not in the target image 502 (e.g., all of the pixels that fall within the resized FOV 508(n) but that are outside the target FOV 504). Thus, the crop component 318 may remove or delete pixels of the aligned image 1002(n) that are not in the target image 502, and such removal or deletion may result in a cropped image 1402(n). That is, the cropped image 1402(n) may be the remaining image of the aligned image 1002(n) after removing any pixels from the aligned image 1002(n) that are not in the target image 502. In this manner, the cropped image 1402(n) may be considered to be the same size as (e.g., have the same number or arrangement of pixels as) the target image 502.
[0105] Note that cropped images 1402(1) through 1402(n) can be considered collectively to form the plurality of cropped images 1402. In various instances, the plurality of cropped images 1402 and the target image 502 can be considered collectively to form the plurality of second edited images 1302.
[0106] In various aspects, each of the set of second edited images 1302 can be considered to contain or exhibit a noticeable or visually noticeable charging artifact oriented along the corresponding scan direction. As a non-limiting example, target image 502 can be captured according to target scan direction 322. As such, target image 502 can exhibit or have a charging artifact that is physically oriented parallel to target scan direction 322. As another non-limiting example, cropped image 1402(1) can be captured according to rotational scan direction 402(1). Thus, target image 1402(1) can exhibit or have a charging artifact that is physically oriented parallel to rotational scan direction 402(1). As yet another non-limiting example, cropped image 1402(n) can be captured according to rotational scan direction 402(n). Thus, cropped image 1402(n) can exhibit or have a charging artifact that is physically oriented parallel to rotational scan direction 402(n). ).
[0107] For non-limiting clarification, reference is made to Figures 15-16.
[0108] FIG. 15 illustrates the above-described aligned image 1202. In various embodiments, reference numeral 1502 can be considered to define a boundary between pixels of the aligned image 1202 that are contained within the target image 502 (and thus within the target FOV 504) and pixels of the aligned image 1202 that are outside the target image 502 (and thus outside the target FOV 504). In various embodiments, pixels that are inside the boundary defined by reference numeral 1502 can be preserved or maintained, while pixels that are outside the boundary can instead be deleted, removed, or cropped. FIG. 16 illustrates the results of such deletion, removal, or cropping. In fact, such deletion, removal, or cropping can be considered to transform the aligned image 1202 into a cropped image 1602 (e.g., one of 1402). As shown, the cropped image 1602 can have the same size (e.g., side length A) as the target image 502 and can show the specimen 304 from the same perspective or orientation as the target image 502. However, the target image 502 may have charging artifacts that are visually parallel to the target scan direction 322. In contrast, the cropped image 1602 may instead have charging artifacts that are visually parallel to the rotational scan direction 702. In other words, although the cropped image 1602 and the target image 502 may show the specimen 304 from the same perspective as each other, the charging artifacts in the cropped image 1602 may be in a different position or in a different spatial distribution than in the target image 502, such that different portions of the specimen 304 may be obscured or difficult to read in the target image 502 than in the cropped image 1602.
[0109] FIG. 17 shows a block diagram of a non-limiting example system including an aggregate image that can facilitate charging artifact mitigation through rotation of the scan direction, according to one or more embodiments described herein.
[0110] In various embodiments, the aggregation component 320 can electronically generate the aggregate image 1702 based on the plurality of second edited images 1302. A non-limiting aspect is described with reference to FIG.
[0111] FIG. 18 illustrates a non-limiting example block diagram illustrating a method for obtaining an aggregate image 1702 according to one or more embodiments described herein.
[0112] In various embodiments, as shown, the aggregation component 320 can electronically aggregate the multiple second edited images 1302 together in any suitable manner, and the result of such aggregation can be referred to as the aggregated image 1702. As a non-limiting example, the aggregated image 1702 can be equal to a pixel-by-pixel average of the target image 502 and the multiple cropped images 1402. In various cases, as described above, each of the multiple second edited images 1302 can have charging artifacts in unique positions or locations that are visually noticeable or prominent. However, the aggregated image 1702, in stark contrast, can be free of charging artifacts, or have negligible, less noticeable, or less prominent charging artifacts. Indeed, each of the multiple second edited images 1302 can be considered to have uniquely or individually located charging artifacts that are not similarly present in other images of the multiple second edited images 1302, such that all such charging artifacts can be canceled or cropped out during averaging.
[0113] As a non-limiting example, the target image 502 may have an inherent or distinct charging artifact parallel to the target scan direction 322. Because no other images can be captured using the target scan direction 322, features or portions of the specimen 304 that are obscured, covered, or distorted by the charging artifact in the target image 502 are not obscured, covered, or distorted by the charging artifact in the remaining images of the plurality of second edited images 1302. Thus, those features or portions of the specimen 304 appear stronger in the aggregate image 1702 because the majority of the plurality of second edited images 1302 depict such features or portions rather than obscuring them.
[0114] As a non-limiting example, cropped image 1402(1) may have an inherent or distinct charging artifact parallel to rotational scan direction 402(1). Because no other images can be captured using rotational scan direction 402(1), features or portions of specimen 304 that are obscured, covered, or distorted by the charging artifact in cropped image 1402(1) are not obscured, covered, or distorted by the charging artifact in the remaining images of the plurality of edited second images 1302. Thus, those features or portions of specimen 304 appear stronger in aggregate image 1702 because the majority of the plurality of edited second images 1302 depict such features or portions rather than obscuring them.
[0115] As yet another non-limiting example, the cropped image 1402(n) may have inherent or distinct charging artifacts parallel to the rotational scan direction 402(n). Because no other images can be captured using the rotational scan direction 402(n), features or portions of the specimen 304 that are obscured, covered, or distorted by the charging artifacts in the cropped image 1402(n) are not clarified, covered, or distorted by the charging artifacts in the remaining images 1302 of the plurality of edited second images. In this manner, those features or portions of the specimen 304 appear stronger in the aggregate image 1702 because the majority of the plurality of edited second images 1302 depict such features or portions rather than obscuring them.
[0116] In this way, regardless of the type of microscope or the type of specimen, charging artifacts can be reduced or attenuated by utilizing rotation of the scan direction.
[0117] 19-20 show non-limiting examples of experimental results according to one or more embodiments described herein.
[0118] In particular, FIG. 19 shows an SEM image 1900 of a ceramic specimen captured using a left-right scan direction. As shown, the SEM image 1900 exhibits noticeable charging artifacts (e.g., noticeable black and white streaks along the left-right direction). Next, FIG. 20 shows an aggregate SEM image 2000 generated in accordance with various embodiments described herein. Specifically, to practice this particular experimental reduction, a total of eight uniformly distributed scan directions (e.g., n+1, n=7, with each pair of adjacent scan directions separated by a 45° angular displacement) were implemented while treating the SEM image 1900 as the target image 502. As can be seen, the aggregate SEM image 2000 does not exhibit noticeable or noticeable charging artifacts or other distortions. As such, these experimental results serve to verify and demonstrate that the various embodiments described herein constitute tangible, real-world technological improvements in the field of charged particle microscopy.
[0119] The scientific instrument systems, methods, or techniques disclosed herein may include interactions with a human user (e.g., via a user local computing device 2320 discussed herein with reference to FIG. 23). These interactions may include providing the user with information (e.g., information regarding the operation of a scientific instrument such as the scientific instrument 2310 of FIG. 23, information regarding an analyzed sample or other test or measurement performed by the scientific instrument, information obtained from a local or remote database, or other information), or providing options for the user to input commands (e.g., to control the operation of a scientific instrument such as the scientific instrument 2310 of FIG. 23 or to control the analysis of data generated by the scientific instrument), queries (e.g., to a local or remote database), or other information. In some embodiments, these interactions may be performed via a graphical user interface (GUI) including a visual display on a display device (e.g., the display device 2210 discussed herein with reference to FIG. 22) that provides output to the user and / or prompts the user to provide input (e.g., via one or more input devices, such as a keyboard, mouse, trackpad, or touchscreen, included in the other I / O devices 2212 discussed herein with reference to FIG. 22). The scientific instrument systems, methods, or techniques disclosed herein may include any suitable GUI for interaction with a user.
[0120] 21 illustrates an example of a graphical user interface 2100 (hereinafter referred to as “GUI 2100”) that can be used in implementing some or all of the assistance methods disclosed herein, according to various embodiments. In various aspects, the GUI 2100 can be provided on any suitable electronic display (e.g., the display device 2210 discussed herein with reference to FIG. 22) of a computing device (e.g., the computing device 2200 discussed herein with reference to FIG. 22) of a scientific instrument assistance system (e.g., the scientific instrument assistance system 2300 described herein with reference to FIG. 23), and a user or technician can interact with the GUI 2100 using any suitable input device (e.g., any of the other I / O devices 2212 discussed herein with reference to FIG. 22) and input technology (e.g., cursor movement, motion capture, facial recognition, gesture detection, voice recognition, button activation).
[0121] GUI 2100 can include a data display area 2102, a data analysis area 2104, a scientific instrument control area 2106, and a settings area 2108. The particular number and arrangement of areas shown in Figure 21 is merely exemplary, and other embodiments of GUI 2100 can include any number and arrangement of areas containing any desired features.
[0122] The data display area 2102 can display data generated by a scientific instrument (eg, the scientific instrument 2310 discussed herein with reference to FIG. 23).
[0123] The data analysis area 2104 can display any suitable data analysis results (e.g., the results of analyzing the data shown in the data display area 2102 or other data). In some embodiments, the data display area 2102 and the data analysis area 2104 can be combined in the GUI 2100 (e.g., to include both data output from a scientific instrument and some analysis of the data in a common graph or area).
[0124] The scientific instrument control area 2106 may include options that allow a user or technician to control scientific instruments (e.g., the scientific instruments 2310 discussed herein with reference to FIG. 23). For example, the scientific instrument control area 2106 may include configurable parameters that govern the operation of such scientific instruments (e.g., configurable parameters that govern the voltage or current of the scientific instrument, that govern the internal temperature of the scientific instrument, or that govern the fluid flow rate of the scientific instrument).
[0125] The settings area 2108 may include options that enable a user or technician to control features and functions of the GUI 2100 (or other GUIs) or perform common computing operations for the data display area 2102 and the data analysis area 2104 (e.g., saving data on a storage device, such as the storage device 2204 discussed herein with reference to FIG. 22 , sending data to another user, labeling data).
[0126] As noted above, the scientific instrument module 102 can be implemented by one or more computing devices. Figure 22 is a block diagram of a computing device 2200 capable of performing some or all of the scientific instrument methods or techniques disclosed herein, according to various embodiments. In some embodiments, the scientific instrument module 102 can be implemented by a single instance of the computing device 2200, or by multiple instances of the computing device 2200. Furthermore, as discussed below, the computing device 2200 (or in the case of multiple computing devices) implementing the scientific instrument module 102 can be part of one or more of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 of Figure 23.
[0127] While computing device 2200 is illustrated as having multiple components, any one or more of these components can be omitted or duplicated as appropriate for the application and configuration. In some embodiments, some or all of the components included in computing device 2200 can be mounted on one or more motherboards and enclosed in a housing (e.g., comprising plastic, metal, or other material). In some embodiments, some of these components can be fabricated on a single system-on-chip (SoC) (e.g., an SoC can include one or more processing units 2202 and one or more storage units 2204). Thus, in various embodiments, computing device 2200 can omit one or more of the components shown in FIG. 22 but can include interface circuitry (not shown) for coupling to one or more of the omitted components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI), a controller area network (CAN) interface, a serial peripheral interface (SPI), an Ethernet interface, a wireless interface, or any other suitable interface). For example, computing device 2200 may omit display device 2210 but may include display device interface circuitry (e.g., connectors and driver circuitry) to which display device 2210 can be coupled.
[0128] Computing device 2200 may include a processing unit 2202 (e.g., one or more processing units). As used herein, the term “processing unit” may refer to any device or part of a device that processes electronic data from registers or memory and converts the electronic data into other electronic data that may be stored in registers or memory. Processing unit 2202 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms in hardware), server processors, or any other suitable processing units.
[0129] Computing device 2200 may include storage device 2204 (e.g., one or more storage devices). Storage device 2204 may include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive bridge RAM (CBRAM) devices), hard drive-based memory devices, solid state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, storage device 2204 may include memory that shares a die with processing unit 2202. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 2204 may include a non-transitory computer-readable medium having instructions thereon that, when executed by one or more processing units (e.g., processing unit 2202), cause computing device 2200 to perform any suitable of the methods or portions of those methods disclosed herein.
[0130] Computing device 2200 may include an interface device 2206 (e.g., one or more instances of interface device 2206). Interface device 2206 may include one or more communications chips, connectors, or other hardware and software to manage communications between computing device 2200 and other computing devices. For example, interface device 2206 may include circuitry to manage wireless communications for transferring data to and from computing device 2200. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, and communications channels that may communicate data through the use of modulated electromagnetic radiation over a non-solid medium. This term does not imply that the associated devices do not include any wiring, although in some embodiments they may not. The circuitry included in interface device 2206 for managing wireless communications may implement any of several wireless standards or protocols, including, but not limited to, Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards (e.g., IEEE 802.16-2005 revision), Institute for Electrical and Electronic Engineers (IEEE) standards including the Long Term Evolution (LTE) project with any amendments, updates, and / or revisions (e.g., Advanced LTE project, Ultra Mobile Broadband (UMB) project (also referred to as “3GPP®2”), etc.). In some embodiments, the circuitry included in interface device 2206 for managing wireless communications may operate in accordance with a Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network.In some embodiments, the circuitry included in the interface device 2206 for managing wireless communications may operate in accordance with Enhanced Data for Evolved GSM (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in the interface device 2206 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolutionary Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 2206 may include one or more antennas (e.g., one or more antenna arrays) for receiving and / or transmitting wireless communications.
[0131] In some embodiments, interface device 2206 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communications protocol. For example, interface device 2206 may include circuitry supporting communications according to Ethernet technology. In some embodiments, interface device 2206 may support both wireless and wired communications, or may support multiple wired or wireless communications protocols. For example, a first set of circuits in interface device 2206 may be dedicated to short-range wireless communications, such as Wi-Fi or Bluetooth, and a second set of circuits in interface device 2206 may be dedicated to long-range wireless communications, such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, a first set of circuits in interface device 2206 may be dedicated to wireless communications, and a second set of circuits in interface device 2206 may be dedicated to wired communications.
[0132] Computing device 2200 may include battery / power circuitry 2208. Battery / power circuitry 2208 may include one or more energy storage devices (e.g., batteries or capacitors) or circuitry for coupling components of computing device 2200 to an energy source separate from computing device 2200 (e.g., AC line power).
[0133] Computing device 2200 may include a display device 2210 (e.g., multiple display devices). Display device 2210 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.
[0134] Computing device 2200 may include other input / output (I / O) devices 2212. The other I / O devices 2212 may include, for example, one or more audio output devices (e.g., speakers, headsets, earphones, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), a location device (e.g., a GPS device that communicates with a satellite-based system to receive the location of computing device 2200), an audio codec, a video codec, a printer, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), an image capture device such as a camera, a keyboard, a cursor control device (e.g., a mouse, stylus, trackball, or touchpad), a barcode reader, a quick response (QR) code reader, or a radio frequency identification (RFID) reader.
[0135] Computing device 2200 may have any suitable form factor for the application and setting, such as a handheld or mobile computing device (e.g., a mobile phone, smartphone, mobile Internet device, tablet computer, laptop computer, netbook computer, ultrabook computer, personal digital assistant (PDA), ultra-mobile personal computer, etc.), a desktop or server computing device, or other network computing component.
[0136] One or more computing devices implementing any of the scientific instrument modules, methods, or techniques disclosed herein may be part of a scientific instrument support system. Figure 23 is a block diagram of an example scientific instrument support system that may perform some or all of the scientific instrument support methods disclosed herein, according to various embodiments. The scientific instrument modules, methods, or techniques disclosed herein (e.g., scientific instrument module 102, computer-implemented method 200, system 306) may be implemented by one or more of the scientific instrument 2310, user local computing device 2320, service local computing device 2330, or remote computing device 2340 of the scientific instrument support system 2300.
[0137] Any of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may include any of the embodiments of computing device 2200, and any of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the form of any suitable one of the embodiments of computing device 2200.
[0138] Each of the scientific instruments 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may include a processing unit 2302, a storage device 2304, and an interface device 2306. The processing unit 2302 may take any suitable form, including any form of the processing unit 2202, and the processing units 2302 included in different ones of the scientific instruments 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the same form or different forms. The storage device 2304 may take any suitable form, including any form of the storage device 2204, and the storage units 2304 included in different ones of the scientific instruments 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the same form or different forms. The interface device 2306 may take any suitable form, including any form of interface device 2206, and the interface device 2306 included in different scientific instruments 2310, user local computing devices 2320, service local computing devices 2330, or remote computing devices 2340 may take the same or different forms.
[0139] The scientific instruments 2310, the user local computing device 2320, the service local computing device 2330, and the remote computing device 2340 can communicate with other elements of the scientific instrument support system 2300 via communication paths 2308. The communication paths 2308 may be communicatively coupled to interface devices 2306 of different ones of the elements of the scientific instrument support system 2300, as shown, and may be wired or wireless communication paths (e.g., according to any of the communication techniques discussed herein with reference to the interface device 2306). While the particular scientific instrument support system 2300 shown in FIG. 23 includes communication paths between each pair of the scientific instruments 2310, the user local computing device 2320, the service local computing device 2330, and the remote computing device 2340, this “fully connected” implementation is merely exemplary, and in various embodiments, various implementations of the communication paths 2308 may not be present. For example, in some embodiments, the service local computing device 2330 may lack a direct communication path 2308 between its interface device 2306 and the interface device 2306 of the scientific instrument 2310, but instead may communicate with the scientific instrument 2310 via a communication path 2308 between the service local computing device 2330 and the user local computing device 2320, and a communication path 2308 between the user local computing device 2320 and the scientific instrument 2310.
[0140] The scientific instrument 2310 may include any suitable scientific instrument, such as a charged particle microscope 302 .
[0141] The user local computing device 2320 can be a computing device local to a user of the scientific instrument 2310 (e.g., according to any of the embodiments of the computing device 2200). In some embodiments, the user local computing device 2320 may also be local to the scientific instrument 2310, although this is not necessary if, for example, a user local computing device 2320 in a user's home or office is remote from but may communicate with the scientific instrument 2310 such that the user may use the user local computing device 2320 to control or access data from the scientific instrument 2310. In some embodiments, the user local computing device 2320 may be a laptop, smartphone, or tablet device. In some embodiments, the user local computing device 2320 may be a portable computing device.
[0142] The service local computing device 2330 can be a computing device (e.g., according to any of the embodiments of computing device 2200) that is local to an entity that provides services to the scientific instrument 2310. For example, the service local computing device 2330 can be local to the manufacturer of the scientific instrument 2310 or to a third-party service company. In some embodiments, the service local computing device 2330 can communicate with the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., via a direct communication path 2308 or via multiple “indirect” communication paths 2308, as discussed above) to receive data regarding the operation of the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., results of self-tests of the scientific instrument 2310, calibration coefficients used by the scientific instrument 2310, measurements of sensors associated with the scientific instrument 2310, etc.). In some embodiments, the service local computing device 2330 may communicate with the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., via a direct communication path 2308 or multiple "indirect" communication paths 2308, as discussed above) to send data to the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., to update programmed instructions such as firmware in the scientific instrument 2310, initiate the performance of a test or calibration sequence in the scientific instrument 2310, update programmed instructions such as software in the user local computing device 2320 or the remote computing device 2340).A user of the scientific instrument 2310 can communicate with the service local computing device 2330 to use the scientific instrument 2310 or the user local computing device 2320 to report problems, request a technician visit to improve the operation of the scientific instrument 2310, order consumables or replacement parts associated with the scientific instrument 2310, or for other purposes.
[0143] The remote computing device 2340 can be a computing device (e.g., according to any of the embodiments of the computing device 2200 discussed herein) that is remote from the scientific instrument 2310 or from the user local computing device 2320. In some embodiments, a data center or other large-scale server environment can include the remote computing device 2340. In some embodiments, the remote computing device 2340 can include network-attached storage (e.g., as part of the storage device 2304). The remote computing device 2340 can store data generated by the scientific instrument 2310, perform analysis of the data generated by the scientific instrument 2310 (e.g., according to programmed instructions), facilitate communications between the user local computing device 2320 and the scientific instrument 2310, or facilitate communications between the service local computing device 2330 and the scientific instrument 2310.
[0144] In some embodiments, one or more of the elements of the scientific instrument support system 2300 shown in Figure 23 can be omitted. Additionally, in some embodiments, multiples of various of the elements of the scientific instrument support system 2300 of Figure 23 may be present. For example, the scientific instrument support system 2300 may include multiple user local computing devices 2320 (e.g., different user local computing devices 2320 associated with different users or in different locations). In another example, the scientific instrument support system 2300 may include multiple scientific instruments 2310 that are all in communication with a service local computing device 2330 and / or a remote computing device 2340. In such an embodiment, the service local computing device 2330 may monitor these multiple scientific instruments 2310, and the service local computing device 2330 may trigger updates or "broadcast" other information to the multiple scientific instruments 2310 simultaneously. Different scientific instruments 2310 in the scientific instrument support system 2300 can be located near each other (e.g., in the same room) or far from each other (e.g., different floors of a building, different buildings, different cities, etc.). In some embodiments, the scientific instruments 2310 can be connected to an Internet of Things (IoT) stack that enables command and control of the scientific instruments 2310 by web-based applications, virtual or augmented reality applications, mobile applications, or desktop applications. Any of these applications can be accessed by a user operating a user local computing device 2320 that is in communication with the scientific instruments 2310 by an intervening remote computing device 2340. In some embodiments, the scientific instruments 2310 can be sold by a manufacturer as part of a local scientific instrument computing unit 2312 along with one or more associated user local computing devices 2320.
[0145] In some embodiments, different ones of the scientific instruments 2310 included in the scientific instrument support system 2300 may be different types of scientific instruments 2310. For example, one scientific instrument 2310 may be a mass spectrometer, while another scientific instrument 2310 may be a chromatograph or an autosampler. In some such embodiments, the remote computing device 2340 or the user local computing device 2320 may combine data from the different types of scientific instruments 2310 included in the scientific instrument support system 2300.
[0146] In various cases, machine learning algorithms or models may be implemented in any suitable manner to facilitate any suitable aspects described herein. To facilitate some of the above machine learning aspects of various embodiments, consider the following artificial intelligence (AI) considerations. Various embodiments described herein may employ artificial intelligence to facilitate automation of one or more features or functions. Components may employ various AI-based schemes to implement various embodiments / examples disclosed herein. To make or assist in the many determinations described herein (e.g., determine, confirm, infer, calculate, predict, prognose, estimate, derive, forecast, detect, calculate), components described herein may examine all or a subset of the data to which they are authorized to access and may infer or determine the state of a system or environment from a series of observations captured via events or data. The determination may be used to identify a particular context or action or may, for example, generate a probability distribution between states. The determination may be probabilistic, i.e., the calculation of a probability distribution between states of interest based on consideration of data and events. The determination may also refer to techniques employed to subsume higher-level events from a series of events or data.
[0147] Such decisions can generate new events or actions from sets of observed events or stored event data, regardless of whether the events are correlated in close temporal proximity and whether the events and data are obtained from one or more event and data sources. The components disclosed herein can employ a variety of classification (explicitly trained (e.g., via training data) and implicitly trained (e.g., via observed behavior, preferences, past information, receipt of exogenous information, etc.)) schemes or systems (e.g., support vector machines, neural networks, expert systems, Bayesian framework networks, fuzzy logic, data fusion engines, etc.) in connection with performing automated or determined actions in connection with the claimed subject matter. Thus, classification schemes or systems can be used to automatically learn and perform multiple functions, actions, or decisions.
[0148] The classifier is f(z) = reliability (class) As such, the input characteristic vector z=(z1,z2,z3,z4,z n ) can be mapped to the confidence that the input belongs to the class. Such classification can use probabilistic or statistical analysis (e.g., consideration of analytical utility and cost) to determine the action to be taken automatically. Support vector machines (SVMs) can be an example of a classifier that can be employed. SVMs operate by finding a hypersurface in the space of possible inputs, which attempts to separate events that trigger the trigger criteria from those that do not. Intuitively, this allows for accurate classification of test data that is close to, but not identical to, the training data. Other direct and indirect model classification approaches can be employed, including, for example, Naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide various patterns of independence. As used herein, classification also includes statistical regression used to create models of priority.
[0149] To provide further context for the various embodiments described herein, Figure 24 and the following discussion are intended to provide a brief, general description of a suitable computing environment 2400 in which various embodiments of the embodiments described herein may be implemented. While the embodiments are described above in the general context of computer-executable instructions that may be executed by one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules, or as a combination of hardware and software.
[0150] Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present invention can be evaluated using other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, and personal computers, portable computing devices, microprocessor-based or programmable consumer electronics devices, each operatively coupled to one or more associated devices.
[0151] The illustrated embodiments of the present specification may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
[0152] A computing device typically includes a variety of media, which may include computer-readable storage media, machine-readable storage media, or communication media, and these two terms are used interchangeably herein as follows. A computer-readable storage medium or machine-readable storage medium may be any available storage medium that can be accessed by a computer, including both volatile and nonvolatile media, and both removable and non-removable media. By way of example, a computer-readable storage medium or machine-readable storage medium may be implemented in connection with any method or technology for storing information, such as, but not limited to, computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.
[0153] A computer-readable storage medium may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray disk (BD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, solid-state drive or other solid-state storage device, or other tangible or non-transitory medium that can be used to store the desired information. In this regard, the terms "tangible" or "non-transitory" as applied to storage medium, memory medium, or computer-readable medium herein exclude only the propagating transitory signal itself as a modifier and do not disclaim any right to all standard storage, memory, or computer-readable medium that do not merely propagate the transitory signal itself.
[0154] The computer-readable storage medium can be accessed by one or more local or remote computing devices, for example, via an access request, query, or other data retrieval protocol, to perform various operations on the information stored by the medium.
[0155] Communication media typically embodies computer-readable instructions, data structures, program modules, or other structured or unstructured data in a data signal, such as a modulated data signal, on a carrier wave or other transport mechanism, and includes any information delivery or transfer media. The term "modulated data signal" or "signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in one or more signals. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
[0156] 24 , an example environment 2400 for implementing various embodiments of the aspects described herein includes a computer 2402, which includes a processing unit 2404, a system memory 2406, and a system bus 2408. The system bus 2408 couples system components, including but not limited to the system memory 2406, to the processing unit 2404. The processing unit 2404 can be any of a variety of commercially available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 2404.
[0157] The system bus 2408 can be any of several types of bus structures that can be further interconnected to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 2406 includes a ROM 2410 and a RAM 2412. The basic input / output system (BIOS) can be stored in non-volatile memory such as a ROM, erasable programmable read-only memory (EPROM), or EEPROM, and contains the basic routines that help to transfer information between elements within the computer 2402, such as during start-up. The RAM 2412 can also include a high-speed RAM such as static RAM for caching data.
[0158] Computer 2402 further includes an internal hard disk drive (HD) 2414 (e.g., EADE, SATA), one or more external storage devices 2416 (e.g., magnetic floppy disk drive (FDD) 2416, memory stick or flash drive reader, memory card reader, etc.), and a drive 2420, such as a solid-state storage device, optical disk drive, or the like, capable of reading from and writing to a disk 2422, such as a CD-ROM disk, DVD, or BD. Alternatively, if a solid-state storage device is included, disk 2422 is not included unless separated. While internal HDD 2414 is shown disposed within computer 2402, internal HDD 2414 could also be configured for external use in a suitable chassis (not shown). Thus, although not shown in environment 2400, a solid-state drive (SSD) could be used in addition to or in place of HDD 2414. The HDD 2414, external storage device(s) 2416, and drive 2420 can be connected to the system bus 2408 by an HDD interface 2424, an external storage interface 2426, and a drive interface 2428, respectively. The interface 2424 for external drive implementations can include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are within consideration of the embodiments described herein.
[0159] The drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of computer 2402, the drives and storage media correspond to the storage of any data in a suitable digital format. While the above description of computer-readable storage media refers to each type of storage device, those skilled in the art will understand that other types of computer-readable storage media, whether currently existing or developed in the future, can be used in the exemplary operating environment, and further, any such storage media can contain computer-executable instructions for performing the methods described herein.
[0160] A number of program modules can be stored in the drives and RAM 2412, including an operating system 2430, one or more application programs 2432, other program modules 2434, and program data 2436. All or portions of the operating system, applications, modules, or data can also be cached in RAM 2412. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.
[0161] Computer 2402 may optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate the hardware environment of operating system 2430, and the emulated hardware may optionally differ from the hardware shown in FIG. 24 . In such an embodiment, operating system 2430 may comprise one of multiple virtual machines (VMs) hosted on computer 2402. Additionally, operating system 2430 may provide a runtime environment, such as the Java Runtime Environment or the .NET Framework, for application 2432. A runtime environment is a consistent execution environment that allows application 2432 to run on any operating system that includes the runtime environment. Similarly, operating system 2430 may support containers, and application 2432 may be in the form of a container, which is a lightweight, standalone, executable package of software that includes, for example, code, runtime, system tools, system libraries, and application configuration.
[0162] Additionally, computer 2402 can be enabled with a security module, such as a trusted processing module (TPM). For example, with a TPM, a boot component can hash the boot component in time and wait for the result to match an assurance value before loading the next boot component. This process can occur at any layer within the code execution stack of computer 2402, for example, applied at the application execution level or the operating system (OS) kernel level, thereby enabling security at any level of code execution.
[0163] A user can enter commands and information into the computer 2402 via one or more wired / wireless input devices, such as a keyboard 2438, a touchscreen 2440, or a pointing device such as a mouse 2442. Other input devices (not shown) can include a microphone, an infrared (IR), radio frequency (RF) or other remote control, a joystick, a virtual reality controller or headset, a game pad, a stylus pen, an image input device such as a camera(s), a gesture sensor input device, a visual motion sensor input device, an emotion or face detection device, a biometric input device such as a fingerprint or iris scanner, etc. These and other input devices are often connected to the processing unit 2404 via an input device interface 2444, which can be coupled to the system bus 2408, but can also be connected by other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH interface, etc.
[0164] A monitor 2446 or other type of display device can also be connected to the system bus 2408 via an interface, such as a video adapter 2448. In addition to the monitor 2446, computers typically include other peripheral output devices (not shown), such as speakers, printers, etc.
[0165] The computer 2402 can operate in a networked environment using logical connections via wired or wireless communications to one or more remote computers, such as remote computer(s) 2450. The remote computer(s) 2450 can be a workstation, a server computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and typically includes many or all of the elements described relative to the computer 2402, although for simplicity, only memory / storage 2452 is shown. The logical connections shown include wired / wireless connections to a local area network (LAN) 2454 or a wide area network, such as, for example, a wide area network (WAN) 2456. Such LAN and WAN networking environments are commonplace in offices and businesses, facilitating enterprise-wide computer networks, such as intranets, all of which can connect to global communications networks, such as the Internet.
[0166] When used in a LAN networking environment, the computer 2402 can be connected to the local network 2454 through a wired or wireless communication network interface or adapter 2458. The adapter 2458 can facilitate wired or wireless communication to the LAN 2454, and the LAN can also include a wireless access point (AP) disposed thereon for communicating with the adapter 2458 in a wireless mode.
[0167] When used in a WAN networking environment, the computer 2402 may include a modem 2460 or may be connected to a communications server on the WAN 2456 via other means for establishing communications across the WAN 2456, such as by way of the Internet. The modem 2460 may be internal or external and a wired or wireless device and may be connected to the system bus 2408 via the input device interface 2444. In a networked environment, program modules depicted for the computer 2402, or portions thereof, may be stored in the remote memory / storage device 2452. It will be understood that the network connections shown are exemplary and other means of establishing a communications link between computers may be used.
[0168] When used in either a LAN or WAN networking environment, computer 2402 can access a cloud storage system or other network-based storage system, such as, but not limited to, a network virtual appliance, that provides one or more aspects of information storage or processing, in addition to or in place of external storage device 2416, as described above. Generally, the connection between computer 2402 and the cloud storage system can be established across LAN 2454 or WAN 2456, for example, by adapter 2458 or modem 2460, respectively. Upon connecting computer 2402 to an associated cloud storage system, external storage interface 2426, with the assistance of adapter 2458 or modem 2460, can manage the storage provided by the cloud storage system in the same way as other types of external storage devices. For example, external storage interface 2426 can be configured to provide access to cloud storage sources such that the sources are physically connected to computer 2402.
[0169] The computer 2402 is operable to communicate with any wireless device or entity operatively positioned for wireless communication, such as a printer, a scanner, a desktop or portable computer, a portable data assistant, a communications satellite, any equipment or location associated with a radio-detectable tag (e.g., a kiosk, a newsstand, a store shelf, etc.), and a telephone. This can include Wireless Fidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, communication can be in a defined structure similar to an existing network, or simply ad-hoc communication between at least two devices.
[0170] FIG. 25 is a schematic block diagram of a sample computing environment 2500 with which the disclosed subject matter can interact. The sample computing environment 2500 includes one or more client(s) 2510. The client(s) 2510 can be hardware or software (e.g., threads, processes, computing devices). The sample computing environment 2500 also includes one or more server(s) 2530. The server(s) 2530 can also be hardware or software (e.g., threads, processes, computing devices). The server(s) 2530 can house threads for performing transformations, for example, by employing one or more embodiments as described herein. One possible communication between the client(s) 2510 and the server(s) 2530 can be in the form of a data packet arranged to be transmitted between two or more computer processes. The sample computing environment 2500 includes a communication framework 2550 that can be employed to facilitate communications between the client(s) 2510 and the server(s) 2530. The client(s) 2510 are operably connected to one or more client data store(s) 2520 that can be employed to store information local to the client(s) 2510. Similarly, the server(s) 2530 are operably connected to one or more server data store(s) 2540 that can be employed to store information local to the server(s) 2530.
[0171] A non-limiting example of an apparatus for implementing various embodiments described herein is shown in FIG. 26. FIG. 26 illustrates a non-limiting example of a dual beam system 2610, which includes a vertically mounted scanning electron microscope (SEM) column and a focused ion beam (FIB) column mounted at an angle of approximately 52° from vertical. Such dual beam systems are commercially available, for example, from FEI Company, Hillsboro, Oregon, the assignee of the present application. However, FIG. 26 illustrates an example of suitable microscope hardware on which various embodiments described herein may be implemented, and it is understood that such microscope hardware is not limited. In other words, various embodiments described herein may be implemented in conjunction with any other suitable type of microscope hardware. The dual beam system 2610 is a non-limiting example of a charged particle microscope 302 or any other scientific instrument described above.
[0172] Scanning electron microscope 2641 can include a dual beam system 2610 along with a power supply and control unit 2645. An electron beam 2643 can be emitted from cathode 2652 by applying a voltage between cathode 2652 and anode 2654. Electron beam 2643 can be focused to a fine spot by condenser lens 2656 and objective lens 2658. Electron beam 2643 can be scanned two-dimensionally over any suitable specimen by deflection coils 2660. Operation of condenser lens 2656, objective lens 2658, or deflection coils 2660 can be controlled by power supply and control unit 2645.
[0173] The electron beam 2643 can be focused onto the substrate 2622, which can be on a movable XY stage 2625 in the lower chamber 2626. When electrons in the electron beam 2643 strike the substrate 2622, secondary electrons can be emitted. These secondary electrons can be detected by a secondary electron detector 2640, as discussed below. A scanning transmission electron microscope (STEM) detector 2662, located below the STEM sample holder 2624 and movable XY stage 2625, can collect electrons transmitted through the sample mounted on the STEM sample holder 2624 as described above.
[0174] The dual beam system 2610 can also include a focused ion beam (FIB) system 2611, which can include a vacuum chamber having an upper neck portion 2612 in which an ion source 2614 and a focusing column 2616 including extraction electrodes and an electrostatic optical system are disposed (in some cases, the upper neck portion can also be referred to as the ion column 2612). The axis of the focusing column 2616 can be tilted 52° (or any other suitable angular displacement) from the axis of the electron column. The ion column 2612 can include an ion source 2614, extraction electrodes 2615, focusing elements 2617, deflection elements 2620, and a focused ion beam 2618. The focused ion beam 2618 can pass from the ion source 2614 through the focusing column 2616 and towards the substrate 2622 between electrostatic deflection means, shown schematically at 2620, which can include, for example, a semiconductor device located on a movable XY stage 2625 in a lower chamber 2626.
[0175] The movable XY stage 2625 can move vertically (along the Z axis) in a horizontal plane (along the X and Y axes). The movable XY stage 2625 can tilt at an angle of approximately 60° and rotate about the Z axis. In some embodiments, a separate STEM sample stage (not shown) can be used. Such a STEM sample stage is movable in the X, Y, and Z axes. A door 2661 can be opened to insert a substrate 2622 onto the XY stage 2625 and to service an internal gas supply reservoir, if one is used. The door 2661 is interlocked so that it cannot be opened when the system is under vacuum.
[0176] An ion pump 2668 can be used to evacuate the neck portion 2612. The chamber 2626 can be evacuated using a turbomolecular and mechanical pumping system 2630 under the control of a vacuum controller 2632. Such a vacuum system can evacuate approximately 1×10 ions within the chamber 2626. -7 Torr ~5×10 -4 A vacuum between 1000 and 1000 Torr can be provided. If an etch-assisting gas, an etch-retarding gas, or a deposition precursor gas is used, the chamber background pressure is typically about 1×10 -5 It may be raised to Torr.
[0177] A high voltage power supply 2634 can provide an appropriate accelerating voltage to electrodes in the focusing column 2616 to energize the focused ion beam 2618. When it strikes the substrate 2622, material can be sputtered (i.e., physically ejected) from the sample. Alternatively, the focused ion beam 2618 can decompose a precursor gas to deposit material.
[0178] A high-voltage power supply 2634 can be connected to the ion source 2614 (which can be a liquid metal ion source) and appropriate electrodes in the ion beam focusing column 2616 to form an approximately 1 keV to 60 keV ion beam 2618 and direct it toward the sample. A deflection controller and amplifier 2636, operating according to a given pattern provided by a pattern generator 2638, can be coupled to deflection elements 2620 (which can be deflection plates), whereby the focused ion beam 2618 can be manually or automatically controlled to track the corresponding pattern on the top surface of the substrate 2622. In some systems, the deflection elements 2620 can be located before the final lens. Beam blanking electrodes (not shown) in the ion beam focusing column 2616 cause the focused ion beam 2618 to impinge on a blanking aperture (not shown) instead of the substrate 2622 when a blanking controller (not shown) applies a blanking voltage to the blanking electrodes.
[0179] The ion source 2614 can provide a metal ion beam, for example, of gallium. Alternatively, the ion source 2614 can be a plasma ion source that extracts ions from a generated plasma. The source can focus a sub-1 / 10 micrometer wide beam at the substrate 2622 for purposes of modifying the substrate 2622 by ion milling, enhanced etching, material deposition, or imaging the substrate 2622.
[0180] A charged particle detector 2640, such as an Everhart-Thornley or multichannel plate, used to detect secondary ion or electron emissions can be connected to a video circuit 2642, which can provide drive signals to a video monitor 2644 and receive deflection signals from the system controller 2619. The location of the charged particle detector 2640 within the lower chamber 2626 can vary in different embodiments. For example, the charged particle detector 2640 can be coaxial with the ion beam and include a hole to allow the ion beam to pass through. In other embodiments, secondary particles can be collected through a final lens and then deflected off-axis for collection.
[0181] The micromanipulator 2647 can precisely move objects within the vacuum chamber. The micromanipulator 2647 may include precision electric motors 2648 located outside the vacuum chamber to provide X, Y, Z, and theta control of a portion 2649 located within the vacuum chamber. The micromanipulator 2647 can be fitted with different end effectors for manipulating small objects. In the embodiment described herein, the end effector can be a thin probe 2650.
[0182] A gas delivery system 2646 can extend into the lower chamber 2626 to introduce and direct a gas vapor toward the substrate 2622. A suitable gas delivery system 2646 is described in commonly assigned U.S. Pat. No. 5,851,413 to Casella et al., entitled "Gas Delivery Systems for Particle Beam Processing." Another gas supply system is described in commonly assigned U.S. Pat. No. 5,435,850 to Rasmussen, entitled "Gas Injection System." For example, iodine can be delivered to enhance etching, or a metal organic compound can be delivered to deposit metal.
[0183] A system controller 2619 can control the operation of various portions of the dual beam system 2610. Through the system controller 2619, a user can scan the focused ion beam 2618 or the electron beam 2643 in a desired manner via instructions entered into any suitable user interface (not shown). Alternatively, the system controller 2619 may control the dual beam system 2610 according to programmed instructions stored in memory 2621. In various embodiments, any of the one or more software components 311 can be implemented or otherwise executed by the system controller 2619.
[0184] Various embodiments may be systems, methods, apparatus, or computer program products at any possible level of technical detail of integration. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to execute aspects of various embodiments. A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media also includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination thereof. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.
[0185] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device. The computer-readable program instructions for carrying out the operations of the various embodiments can be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and procedural programming languages such as the "C" programming language or similar languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the computer, and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider).In some embodiments, electronic circuitry, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to individualize the electronic circuitry to perform various aspects.
[0186] Various aspects are described herein with reference to flowchart or block diagrams of methods, apparatus (systems), and computer program products according to various embodiments. It will be understood that each block of the flowchart or block diagrams, and combinations of blocks in the flowchart or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, whereby the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the function(s) / act(s) specified in the flowchart or block diagram block(s). These computer-readable program instructions can also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, or other device to function in a particular manner, whereby the computer-readable storage medium on which the instructions are stored includes an article of manufacture containing instructions that implement aspects of the function(s) / act(s) specified in the flowchart or block diagram block(s). The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to perform a series of operational acts on the computer, other programmable apparatus, or other device to generate a computer-implemented process, whereby the instructions executing on the computer, other programmable apparatus, or other device implement the function / acts identified in a block or blocks of the flowchart or block diagram.
[0187] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the identified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. Furthermore, each block of the block or flowchart diagrams, and combinations of blocks in the block or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions.
[0188] Although the present subject matter has been described above in the general context of computer-executable instructions for a computer program product executed by one or more computers, those skilled in the art will recognize that the disclosure can be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that various aspects can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputer devices, mainframe computers, as well as computers, handheld computing devices (e.g., PDAs, phones), microprocessor-based or programmable consumer or personal electronic devices, etc. The illustrated aspects can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked through a communications network. However, all or part of the aspects of the present disclosure can be implemented on stand-alone computers. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
[0189] As used herein, the terms "component," "system," "platform," "interface," and the like may refer to or include computer-related entities or entities associated with an operating machine having one or more specific functions. The entities disclosed herein can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, or a computer. By way of illustration, both an application running on a server and the server can be components. One or more components can reside within a process or thread of execution, and components can be local to one computer or distributed between two or more computers. In another example, each component can execute from various computer-readable media having various data structures stored thereon. Components can communicate via local or remote processes, for example, according to signals having one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, or interacting across a network such as the Internet via signals). As another example, a component may be a device having a particular function provided by mechanical parts operated by electrical or electronic circuitry that is operated by a software or firmware application executed by a processor. In such cases, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application.As yet another example, a component may be a device that provides a particular function without mechanical parts through electronic components, where the electronic components may include a processor or other means for executing software or firmware that at least partially provides the functionality of the electronic component. In one aspect, a component may emulate the electronic component via a virtual machine, for example, in a cloud computing system.
[0190] Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from context, "X uses A or B" is intended to mean any natural inclusive permutation. That is, if X uses A, if X uses B, or if X uses both A and B, then "X uses A or B" is satisfied by any of the foregoing. As used herein, the term "and / or" is intended to have the same meaning as "or." Furthermore, the articles "a" and "an," as used in this specification and the accompanying drawings, should generally be interpreted to mean "one or more" unless otherwise specified or clear from context as indicating the singular form. As used herein, the terms "example" or "exemplary" are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter described herein is not limited by such examples. Furthermore, any aspect or design described herein as an "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, and is not meant to exclude equivalent exemplary structures and techniques known to those skilled in the art.
[0191] The disclosure herein describes non-limiting examples. For ease of explanation and illustration, various parts of the specification utilize the terms "each," "every," or "all" when discussing various examples. The use of the terms "each," "each," or "all" is not limiting. That is, when the disclosure herein provides a description that applies to "each," "each," or "all" of a particular object or component, it should be understood that this is a non-limiting example, and it should be further understood that in various other examples, it may apply less frequently than "each," "each," or "all" of that particular object or component.
[0192] As used in the subject specification, the term "processor" refers to virtually any computing processing unit or device, including, but not limited to, a single processor with software multithreading, a multicore processor, a multicore processor with software multithreading, a multicore processor with hardware multithreading, a parallel platform, or a parallel platform using distributed shared memory. Furthermore, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, a processor can utilize nanoscale architectures, such as, but not limited to, molecular and quantum dot-based transistors, switches, and gates, to optimize space usage or improve the performance of user devices. A processor can also be implemented as a combination of computing processing units. In this disclosure, terms such as "store," "storage," "data store," "data storage," "database," and substantially any other information storage component associated with the operation and functionality of a component are utilized to refer to a "memory component," an entity embodied in a "memory," or a component that includes a memory. It should be understood that the memory or memory components described herein can be either volatile or non-volatile memory, or can include both volatile and non-volatile memory.By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory may include, for example, RAM, which may act as external cache memory. By way of example, and not limitation, RAM is available in many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the memory components of the systems or computer-implemented methods described herein are intended to comprise, but are not limited to, these and any other suitable types of memory.
[0193] The foregoing includes merely example systems and computer-implemented methods. Of course, for purposes of describing this disclosure, it is not possible to describe every conceivable combination of components or computer-implemented method, although many more combinations and variations of the present disclosure are possible. Furthermore, to the extent that terms such as "includes," "has," "possesses," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be as inclusive as the term "comprising," as "comprising" is interpreted when used as a transitional term in a claim.
[0194] The descriptions of various embodiments are presented for illustrative purposes, but are not intended to be exhaustive or limited to the embodiments of the present disclosure. Many modifications and variations will be apparent without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the market, or to enable those skilled in the art to understand the embodiments described herein.
[0195] Various non-limiting aspects are illustrated in the following examples.
[0196] Example 1: A system can include a processor that executes computer-executable components stored in a non-transitory computer-readable memory, the computer-executable components can include an access component that can access a charged particle microscope on which a specimen is mounted, and an aggregation component that can generate an aggregate image of the specimen based on multiple images of the specimen captured by the charged particle microscope according to a target scan direction and multiple rotational scan directions.
[0197] Example 2: A system according to any of the preceding examples can be implemented in which the plurality of rotational scan directions and the target scan direction can be uniformly distributed within a 360° range.
[0198] Example 3: The system described in any preceding example can be implemented where the plurality of rotational scan directions and the target scan direction can collectively have an even cardinality, whereby for each first scan direction there can be a second scan direction that points 180° away from the first scan direction.
[0199] Example 4: A system described in any preceding example can be implemented in which a target image of the plurality of images can be captured according to the target scan direction and can have a target field of view of the specimen, and the remaining images of the plurality of images can each be resized to have a field of view circumscribing the target field of view.
[0200] Example 5: A system described in any preceding example can be implemented in which the computer-executable component can further include an alignment component that can correctively rotate each of the plurality of images according to the plurality of rotational scan directions, thereby generating a plurality of aligned images aligned at the target field of view.
[0201] Example 6: A system according to any of the preceding examples can be implemented in which the alignment component can apply a drift correction to each of the plurality of aligned images.
[0202] Example 7: A system described in any preceding example can be implemented where the computer-executable component can further include a crop component that crops out any pixels or voxels that are not in the target field of view outside the plurality of aligned images after drift correction, respectively, thereby resulting in a plurality of cropped images that are the same size as the target images.
[0203] Example 8: The system described in any of the preceding examples can be implemented where the aggregation component can average the multiple cropped images and the target image together, thereby generating the aggregate image.
[0204] Example 9: The system described in any preceding example can be implemented where the specimen can become non-uniformly charged during scanning, where each of the plurality of images can exhibit a respective charging artifact, and where the aggregate image can exhibit suppressed charging artifacts.
[0205] In various embodiments, any combination(s) of Examples 1-9 can be implemented.
[0206] Example 10: A computer-implemented method may include accessing a charged particle microscope with a specimen mounted thereon by a device operatively coupled to a processor, and generating, by the device, an aggregate image of the specimen based on a plurality of images of the specimen captured by the charged particle microscope according to a target scan direction and a plurality of rotational scan directions.
[0207] Example 11: The computer-implemented method according to any preceding example can be implemented, wherein the plurality of rotational scan directions and the target scan direction can be uniformly distributed within a 360° range.
[0208] Example 12: The computer-implemented method of any preceding example can be implemented, wherein the plurality of rotational scan directions and the target scan direction collectively have an even cardinality, whereby for each first scan direction there can be a second scan direction that points 180° away from the first scan direction.
[0209] Example 13: The computer-implemented method of any preceding example can be implemented, wherein a target image of the plurality of images can be captured according to the target scan direction and can have a target field of view of the specimen, and the remaining images of the plurality of images can each be resized to have a field of view circumscribing the target field of view.
[0210] Example 14: The computer-implemented method described in any preceding example can be implemented, further comprising correctively rotating each of the plurality of images according to the plurality of rotational scan directions by the device, thereby generating a plurality of aligned images aligned with the target field of view.
[0211] Example 15: The device may implement the computer-implemented method of any preceding example, further comprising applying a drift correction to each of the plurality of aligned images.
[0212] Example 16: The computer-implemented method of any preceding example may be implemented, further comprising cropping, by the device and from the plurality of aligned images, respectively, any pixels or voxels that are not present in the target field of view, thereby generating a plurality of cropped images, each having the same size as the target image.
[0213] Example 17: The computer-implemented method of any preceding example may be implemented, further comprising averaging, by the device, the plurality of cropped images and the target image together, thereby generating the aggregate image.
[0214] Example 18: The computer-implemented method of any preceding example can be implemented, wherein the specimen can be non-uniformly charged during scanning, each of the plurality of images can exhibit a respective charging artifact, and the aggregate image can exhibit a suppressed charging artifact.
[0215] In various embodiments, any combination(s) of Examples 10-18 can be implemented.
[0216] Example 19: A computer program product for facilitating mitigation of charging artifacts through rotation of scan direction may include a non-transitory computer-readable memory having program instructions embedded therein. In various aspects, the program instructions may be executable by a processor to cause a scanning electron microscope having a specimen mounted thereon to capture a plurality of images of the specimen according to a target scan direction and a plurality of rotational scan directions, each of the images exhibiting a respective charging artifact, to perform a corrective rotation of the plurality of images captured according to the plurality of rotational scan directions, thereby aligning the plurality of images with the target scan direction, and, after the corrective rotation, to average the plurality of images, thereby generating an aggregate image of the specimen, wherein the visibility of the charging artifact in the aggregate image may be lower than the visibility of each of the charging artifacts in the plurality of images.
[0217] Example 20: The computer program product according to any preceding example can be implemented, wherein the target scan direction and the plurality of rotational scan directions can be uniformly distributed within a range of 360°.
[0218] In various embodiments, any combination(s) of Examples 19-20 can be implemented.
[0219] In various embodiments, any combination(s) of Examples 1-20 can be implemented.
Claims
1. 1. A system comprising: a processor that executes computer-executable components stored in non-transitory computer-readable memory; The computer-executable components include: an access component for accessing the charged particle microscope on which the specimen is mounted; an aggregation component that generates an aggregate image of the specimen based on a plurality of images of the specimen captured by the charged particle microscope according to a target scan direction and a plurality of rotational scan directions.
2. The system of claim 1 , wherein the plurality of rotational scan directions and the target scan direction are uniformly distributed within a 360° range.
3. 3. The system of claim 2, wherein the plurality of rotational scan directions and the target scan direction together have an even cardinality, whereby for every first scan direction there is a second scan direction oriented 180 degrees away from the first scan direction.
4. 2. The system of claim 1, wherein a target image of the plurality of images is captured according to the target scan direction and has a target field of view of the specimen, and the remaining images of the plurality of images have respective resized fields of view surrounding the target field of view.
5. the computer-executable components: The system of claim 4 , further comprising an alignment component that performs a corrective rotation of each of the plurality of images according to the plurality of rotational scan directions, thereby generating a plurality of aligned images aligned with the target field of view.
6. The system of claim 5 , wherein the alignment component applies a drift correction to each of the aligned images.
7. the computer-executable components:
7. The system of claim 6, further comprising a crop component that, after drift correction, crops out of the plurality of aligned images any pixels or voxels that are not in the target field of view, thereby generating a plurality of cropped images, each having the same size as the target image.
8. The system of claim 7 , wherein the aggregation component averages the plurality of cropped images and the target image together, thereby generating the aggregate image.
9. 2. The system of claim 1, wherein the specimen is non-uniformly charged during scanning, and wherein each of the plurality of images exhibits a respective charging artifact, and the aggregate image exhibits a suppressed charging artifact.
10. 1. A computer-implemented method comprising: a device operatively coupled to the processor accessing a charged particle microscope with a specimen mounted thereon; and generating an aggregate image of the specimen based on a plurality of images of the specimen captured by the charged particle microscope according to a target scan direction and a plurality of rotational scan directions.
11. The computer-implemented method of claim 10 , wherein the plurality of rotational scan directions and the target scan direction are uniformly distributed within a 360° range.
12. 12. The computer-implemented method of claim 11, wherein the plurality of rotational scan directions and the target scan direction together have an even cardinality, whereby for every first scan direction there is a second scan direction oriented 180 degrees away from the first scan direction.
13. 11. The computer-implemented method of claim 10, wherein a target image of the plurality of images is captured according to the target scan direction and has a target field of view of the specimen, and the remaining images of the plurality of images have respective resized fields of view surrounding the target field of view.
14. The computer-implemented method of claim 13 , further comprising the step of: the device correctively rotating each of the plurality of images according to the plurality of rotational scan directions, thereby generating a plurality of aligned images aligned with the target field of view.
15. The computer-implemented method of claim 14 , further comprising the device applying a drift correction to each of the aligned images.
16. 16. The computer-implemented method of claim 15, further comprising the device cropping out of the plurality of aligned images any pixels or voxels that are not in the target field of view, respectively, thereby generating a plurality of cropped images, each having the same size as the target image.
17. The computer-implemented method of claim 16 , further comprising the device averaging the plurality of cropped images and the target image together, thereby generating the aggregate image.
18. 11. The computer-implemented method of claim 10, wherein the specimen becomes non-uniformly charged during scanning, and wherein each of the plurality of images exhibits a respective charging artifact, and wherein the aggregate image exhibits a suppressed charging artifact.
19. 1. A computer program for facilitating charging artifact mitigation via rotation of scan direction, comprising: the computer program comprising a non-transitory computer-readable memory having program instructions embodied therein; the program instructions are executable by a processor; access to a scanning electron microscope equipped with a specimen; causing the scanning electron microscope to capture a plurality of images of the specimen according to a target scan direction and a plurality of rotational scan directions, each of the images showing a respective charging artifact; correcting and rotating the charging artifacts of each of the plurality of images captured according to the plurality of rotational scan directions so that the plurality of images are aligned with the target scan direction; and averaging the plurality of images after the corrective rotation to thereby generate an aggregate image of the specimen, wherein the visibility of charging artifacts in the aggregate image is lower than the visibility of each of the charging artifacts in the plurality of images.
20. 20. The computer program of claim 19, wherein the target scan direction and the plurality of rotational scan directions are uniformly distributed within a 360° range.