Teaching charged particle microscope robot gripper via beam splitting method

Abstract

The invention relates to an apparatus for facilitating the teaching of a charged particle microscope robot gripper via a beam splitting method. In various embodiments, a system may access a charged particle microscope having a vacuum chamber, where the vacuum chamber may include a robotic gripper and a microscope grid container. In various aspects, the system may teach the robotic gripper to angularly or translationally align its axis-axis with an aperture-axis of the microscope grid container based on a beam splitter equipped light source coupled to the robotic gripper. The invention also relates to a computer-implemented method and a computer program product.

Description

Technical Field

[0001] This invention relates to an apparatus for facilitating the teaching of a charged particle microscope robotic gripper via beam splitting. The invention also relates to a computer-implemented method and a computer program product. Background Technology

[0002] Aligning the charged particle microscope robot gripper with the grid container can be difficult. Summary of the Invention

[0003] The following summary is presented to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or essential elements, or to depict any scope of a particular embodiment or any scope of the claims. Its sole purpose is to present the concepts in a simplified form as a prelude to the more detailed description that follows. In one or more embodiments described herein, methods, apparatuses, or computer program products that facilitate teaching a charged particle microscope robotic gripper via beam splitting are described.

[0004] According to one or more embodiments, an apparatus is provided. In various aspects, the apparatus may include a charged particle microscope having a vacuum chamber, wherein the vacuum chamber may include a robotic gripper and a microscope mesh container. In various instances, the apparatus may include a processor configured to teach the robotic gripper to align its axis-to-axis with the aperture-to-axis of the microscope mesh container angularly or translationally, based on a light source coupled to the robotic gripper.

[0005] According to one or more embodiments, a computer-implemented method is provided. In various aspects, the computer-implemented method may include: accessing a charged particle microscope having a vacuum chamber via a device operatively coupled to a processor, wherein the vacuum chamber may include a robotic gripper and a microscope mesh container. In various instances, the computer-implemented method may include: through the device, teaching the robotic gripper to align its axis-to-axis with the aperture-to-axis of the microscope mesh container angularly or translationally, based on a light source coupled to the robotic gripper.

[0006] According to one or more embodiments, a computer program product is provided for facilitating the teaching of a robotic gripper for a charged particle microscope via beam splitting. In various embodiments, the computer program product may include a non-transitory computer-readable storage memory having program instructions embodied therein. In various aspects, the program instructions are executable by a processor to enable the processor to access a charged particle microscope having a vacuum chamber, wherein the vacuum chamber may include a robotic gripper and a microscope mesh container. In various instances, the program instructions are executable by a processor to cause a light beam emitted by a light source attached to the robotic gripper to pass through a beam splitter, wherein the beam splitter is configured to direct a first portion of the beam toward a camera, wherein the beam splitter is configured to direct a second portion of the beam toward a reflector positioned on or within an aperture in the microscope mesh container, wherein the reflector is configured to reflect the second portion of the beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the beam toward the camera. In various cases, the program instructions can be executed by the processor to cause the processor to access an image captured by a camera, wherein the image depicts a first illumination spot caused by a first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portions of the light beam. In various aspects, the program instructions can be executed by the processor to cause the processor to identify one or more angular or translational movements of the robotic gripper that would align the robot gripper's axis-to-axis with the aperture of the microscope mesh container, either angularly or translationally, based on the distance between the respective centroids of the first and second illumination spots. In various instances, the program instructions can be executed by the processor to cause the processor to perform one or more angular or translational movements of the robotic gripper. Attached Figure Description

[0007] Various embodiments will be readily understood through the following detailed description taken in conjunction with the accompanying drawings. For ease of description, the same reference numerals indicate the same structural elements. The embodiments are shown in the figures by way of example rather than limitation. These figures are not necessarily drawn to scale.

[0008] Figure 1 Exemplary non-limiting block diagrams of scientific instrument modules according to various embodiments described herein are shown.

[0009] Figure 2 An exemplary non-limiting flowchart of a computer-implemented method according to various embodiments described herein is shown.

[0010] Figure 3 A block diagram of an exemplary non-limiting system according to one or more embodiments described herein is shown, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0011] Figures 4 to 12 Exemplary non-limiting block diagrams are shown according to one or more embodiments described herein, illustrating how a robotic gripper and mesh container for a charged particle microscope can be fitted or equipped with alignment hardware.

[0012] Figure 13 A block diagram of an exemplary, non-limiting system including images is shown according to one or more embodiments described herein, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0013] Figures 14 to 17 Non-limiting examples of images according to one or more embodiments described herein are shown.

[0014] Figure 18 A block diagram of an exemplary non-limiting system including an object detection algorithm and overlap determination according to one or more embodiments described herein is shown, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0015] Figure 19 Exemplary non-limiting block diagrams are shown according to one or more embodiments described herein, illustrating how overlap determination can be obtained from an image.

[0016] Figure 20 A block diagram of an exemplary non-limiting system including a novel angular configuration according to one or more embodiments described herein is shown, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0017] Figures 21 to 26 Exemplary non-limiting block diagrams are shown according to one or more embodiments described herein, illustrating how new angular configurations can be identified based on overlap determination.

[0018] Figures 27 to 30 A flowchart of an exemplary non-limiting computer-implemented method according to one or more embodiments described herein is shown, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0019] Figure 31 Exemplary non-limiting block diagrams are shown according to one or more embodiments described herein, illustrating how various artificial intelligence models can be trained.

[0020] Figures 32 to 36 Exemplary non-limiting images of various experimental practices according to one or more embodiments described herein are shown.

[0021] Figure 37Exemplary non-limiting block diagrams of graphical user interfaces according to various embodiments described herein are shown, which can be used to perform some or all of the methods or techniques disclosed herein.

[0022] Figure 38 Exemplary non-limiting block diagrams of computing devices according to various embodiments described herein are shown, which may perform some or all of the methods or techniques disclosed herein.

[0023] Figure 39 Exemplary non-limiting block diagrams of scientific instrument support systems according to various embodiments described herein are shown, in which some or all of the methods or techniques disclosed herein may be performed.

[0024] Figure 40 A block diagram of an exemplary, non-limiting operating environment is shown, which may facilitate one or more embodiments described herein.

[0025] Figure 41 An exemplary networking environment capable of operating to perform the various specific implementations described herein is shown.

[0026] Figure 42 Exemplary dual-beam microscopes that can be implemented according to various embodiments described herein are shown. Detailed Implementation

[0027] The following detailed description is illustrative only and is not intended to limit the implementation and / or application or use of the embodiments. Furthermore, there is no intention to be bound by any express or implied information presented in the foregoing Background or Summary of the Invention or Detailed Description sections.

[0028] One or more embodiments will now be described with reference to the accompanying drawings, wherein the same reference numerals are used throughout to refer to the same elements. In the following description, numerous specific details are set forth for purposes of explanation in order to provide a more thorough understanding of one or more embodiments. However, it will be apparent, in various cases, that one or more embodiments may be practiced without these specific details.

[0029] The various operations can be described sequentially as multiple discrete actions or operations in a manner most conducive to understanding the subject matter disclosed herein. However, the described order should not be construed as implying that these operations must depend on the order. In particular, these operations may be performed in an order different from the order presented. The described operations may be performed in an order different from the described embodiments. Various additional operations may be performed, or the described operations may be omitted in additional embodiments.

[0030] While some elements may be represented in the singular (e.g., "processing device"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented as different operations of those operations being performed by different processing devices. As used herein, the phrase "based on" should be understood to mean "at least partially based on," unless otherwise specified.

[0031] Charged particle microscopes (e.g., scanning electron microscopes (SEM), transmission electron microscopes (TEM), dual-beam microscopes) can be any suitable computerized device capable of capturing or generating microscopic or nanoscale images in research, laboratory, or clinical operating environments. To facilitate the capture or generation of such images, charged particle microscopes can utilize complex arrangements of drivable components (e.g., ion sources, electron sources, optical lenses or apertures, optical plates or deflectors, microscope tubes, coils, heaters, coolers, fluid valves, fluid pumps, circuit switches, specimen stages), sensors (e.g., ion detectors, electron detectors, voltmeters, thermistors, potentiometers, pressure gauges), or consumables (e.g., carrier liquids, calibrators, filters, reactive gases).

[0032] To enable a charged particle microscope to capture images of specimens (e.g., integrated circuit chips, semiconductor wafers, thin films, organic or biological tissue samples), the specimen can be placed on an actuable stage within the vacuum chamber of the charged particle microscope. This placement can be accomplished via a robotic gripper and a grid container, also located within the vacuum chamber (e.g., within a load-locking section of the vacuum chamber). Specifically, the grid container can be any suitable fixed, static, or otherwise substantially static structure that can physically hold, contain, or otherwise hold any suitable number of grids, each of which can be any suitable stencil, carrier, substrate, or culture dish on which multiple specimens can be physically supported. Furthermore, the robotic gripper can be any suitable articulated or telescopic end effector (e.g., an automated gripper, automated clamp, automated hand) that can selectively grasp one or more desired grids from the grid container and thereby grasp one or more desired specimens, and can transport or transfer one or more desired grids to the actuable stage by angular or translational movement. Therefore, the robotic gripper can release its grip on one or more desired grids, thereby placing, arranging, or retaining one or more desired grids and thus one or more desired specimens on an actuable stage, and thus the charged particle microscope can subsequently scan or capture images of one or more desired specimens. After such scanning or image capture, the robotic gripper can grasp one or more desired grids from the actuable stage and can move angularly or translatively to transport or transfer one or more desired grids back to the grid container. Thus, the robotic gripper can release its grip on one or more desired grids, thereby allowing one or more desired grids and thus one or more desired specimens to be placed back or repositioned on or within the grid container.

[0033] For any given mesh, the mesh can be physically held, contained, or otherwise supported by the mesh container based on or due to the insertion of an axis protruding from the given mesh (e.g., from the mesh itself, or from any suitable intermediate support, carrier, core, or other attachment temporarily or permanently coupled to the given mesh) into the corresponding hole of the mesh container. In effect, the axis and the hole can be considered mating surfaces. Therefore, when an axis protruding from the given mesh is physically inserted into the hole of the mesh container, the given mesh can be considered to be removably or slidably mating with the mesh container. Thus, when it is necessary to transport or transfer the given mesh from the mesh container to an actuable stage, the robotic gripper can grasp the given mesh (or any intermediate support, carrier, core, or other attachment temporarily or permanently coupled to the given mesh, as applicable) and can physically advance the given mesh to slide the axis protruding from the given mesh out of the hole, thereby releasing the given mesh from the mesh container. Conversely, when a given mesh needs to be transported or transferred from an actuable stage to a mesh container, the robotic gripper can grasp the given mesh (or any intermediate support, carrier, core, or other attachment that is temporarily or permanently coupled to the given mesh, if applicable) and can physically advance the given mesh so that it slides into a hole from the axis protruding from the given mesh, thereby allowing the given mesh to removably or slidably engage with the mesh container again.

[0034] To enable a robotic gripper to correctly, properly, or otherwise accurately retrieve a given mesh from or place it back into a mesh container, the gripper should know how to position or orient itself angularly or translatively in three-dimensional space such that the axis projecting from the mesh container is precisely aligned with the hole in the mesh container in two translational directions (e.g., left-right, up-down) and two rotational directions (e.g., pitch, yaw). In practice, the gap between the axis and the hole may be as small as 6 to 30 micrometers. If the axis is not precisely aligned with the hole when the gripper attempts to remove a given mesh from or place it back into the mesh container, the outer surface of the axis may experience excessive friction, jamming, or impact with the inner surface of the hole. In some cases, such excessive friction, jamming, or impact may cause physical compression, shaking, or other damage to any specimen on the given mesh, which may be undesirable. In other cases, such excessive friction, jamming, or impact may generate metal shavings or other particles that could contaminate any specimens on a given mesh, which is undesirable. In still other cases, such excessive friction, jamming, or impact may cause physical degradation or premature wear of the robot gripper, the given mesh, or the mesh container, which is also undesirable.

[0035] Unfortunately, as the inventors of the various embodiments described herein recognize, prior art for teaching a robotic gripper how to translate or angularly align an axis with a hole is imprecise and time-consuming. In practice, prior art utilizes a pipe mirror or camera to attempt visual observation of when the axis is translated or angularly aligned with the hole. Some prior art utilizes a pipe mirror or camera physically coupled to or held by the robotic gripper to capture a video stream showing the appearance of the hole as viewed from the axis's angle. Therefore, any three-dimensional position or orientation of the robotic gripper that makes the hole in the video stream appear, or seem, translated or angularly aligned can be considered, marked, or identified as aligned. Conversely, any three-dimensional position or orientation of the robotic gripper that makes the hole in the video stream appear, or seem, not translated or angularly aligned can be considered, marked, or identified as not aligned. Such prior art is imprecise because it can be considered a "visual" method, only providing a rough estimate of the alignment position. Furthermore, most available cameras or pipe mirrors are large or bulky compared to specimen grids (e.g., a grid or grid holder can weigh on the order of 1.3 grams, while most available miniature cameras or pipe mirrors weigh on the order of tens of grams). Therefore, such prior art suffers from additional inaccuracies because the robotic gripper is deflected more by the weight of the camera or pipe mirror than by the weight of the grid or grid holder (e.g., due to this additional deflection, the position of the robotic gripper, which appears visually aligned with the hole when the robotic gripper (as an alternative) holds the camera or pipe mirror, may actually not align the axis with the hole). Other prior art attempts to address this additional deflection problem by using cameras or pipe mirrors that are not coupled to or held by the robotic gripper and in which both the robotic gripper and the hole are within its field of view. While other implementations of this kind eliminate the problem of additional deflection, they require multiple cameras or pipe mirrors (e.g., top-view or bottom-view cameras or pipe mirrors perform poorly in showing vertical translation alignment and pitch alignment; side-view cameras or pipe mirrors perform poorly in showing horizontal translation alignment and yaw alignment), and they still fall into the category of imprecise "visual" methods, providing only coarse alignment estimates. In any case, existing technologies relying on cameras or pipe mirrors require hours or even days of meticulous position iterations to identify which 3D positions or orientations of the robotic gripper appear visually aligned with the holes in the mesh container.

[0036] Therefore, it may be desirable to have a system or technology that can teach a robotic gripper how to align with the holes in a mesh container more efficiently or effectively.

[0037] The various embodiments described herein address this technical problem. One or more embodiments described herein may include: systems, computer-implemented methods, apparatus, or computer program products that facilitate teaching a charged particle microscope robotic gripper via beam splitting. In other words, the inventors of the various embodiments described herein have designed various techniques for teaching a robotic gripper how to become angularly or translationally aligned with the aperture of a mesh container, techniques that do not suffer from the inaccurate or time-consuming iteration problems of prior art. Such teaching can be facilitated via a light source coupled to or held by the robotic gripper. In particular, the light source may emit a beam of light through a beam splitter, which is also coupled to or held by the robotic gripper. In various aspects, the beam splitter may direct a first portion of the beam toward a camera. In various instances, the beam splitter may direct a second portion of the beam toward a reflector positioned on or within the aperture of the mesh container. In various cases, the reflector may direct the second portion of the beam back to the beam splitter, and the beam splitter may direct at least some of the second portion of the beam toward the camera. Therefore, the camera can capture an image depicting two illumination spots: a first spot caused by a first portion of the beam; and a second spot caused by at least some of the second portion of the beam.

[0038] In various aspects, the image can be used to determine whether the robot gripper is angularly aligned with the hole in the mesh container. For example, if the first and second light spots are not concentric, it can be concluded that the robot gripper is not angularly aligned with the hole (e.g., it can be concluded that the robot gripper is not aligned with the hole in the yaw or pitch direction). Conversely, if the first and second light spots are concentric, it can (alternatively) be concluded that the robot gripper is angularly aligned with the hole (e.g., it can be concluded that the robot gripper is aligned with the hole in both the yaw and pitch directions).

[0039] In various other respects, the image can also be used to determine whether the robotic gripper is translationally aligned with the hole in the mesh container. In practice, the reflector can be constructed non-uniformly or segmentally, such that it provides the strongest or highest amount of reflection at its center (and thus at the center of the hole), and that it provides a gradually decreasing or weakening amount of reflection towards its edges (and thus towards the inner surface of the hole). Therefore, if the intensity of the second light spot is weak (e.g., below a threshold), it can be concluded that the robotic gripper is not translationally aligned with the hole (e.g., it can be concluded that the robotic gripper is not aligned vertically or horizontally relative to the hole, such that the light source is not pointing towards the center of the hole). Conversely, if the intensity of the second light spot is strong (e.g., above a threshold), it can be (alternatively) concluded that the robotic gripper is translationally aligned with the hole (e.g., it can be concluded that the robotic gripper is aligned both vertically and horizontally relative to the hole, such that the light source is pointing towards the center of the hole).

[0040] In any case, the various embodiments described herein can teach a robotic gripper without incurring the disadvantages of prior art. In particular, acrylonitrile-level light sources (e.g., microlasers, miniature light-emitting diodes (LEDs)) and acrylonitrile-level beam splitters are more readily available than acrylonitrile-level cameras or channel mirrors. Therefore, the various embodiments described herein avoid causing the robotic gripper to experience significantly greater deflections during the learning or teaching phase than it does when holding the mesh. Furthermore, the various embodiments described herein do not require the use of multiple cameras or channel mirrors. Instead, a single camera or channel mirror (in which the beam splitter guides the light) is sufficient. Moreover, at least regarding angular alignment, the various embodiments described herein do not require as many iterations as the "visual" methods of the prior art. In fact, since the dimensions of the robotic gripper and the mesh container, as well as the dimensions between them, are known or controlled, kinematic formulas can be derived to mathematically correlate the three-dimensional position or orientation of the robotic gripper with the in-image positions of the first and second light spots in the images captured by the cameras. Therefore, if the first and second light spots are not yet concentric in the image, these kinematic formulas and the current in-image positions of the first and second light spots can be used to identify which 3D position or orientation of the robot gripper will make the first and second light spots concentric (or at least closer to concentricity). Once the first and second light spots reach a threshold level of concentricity (e.g., once they are within a threshold margin or proximity to each other), angular alignment can be further achieved by fitting a minimum area profile around both the first and second light spots; having the robot gripper sweep a series of nearby positions or orientations; and selecting any sweep position or orientation that minimizes the area value of the minimum area profile. Thus, the time-consuming trial-and-error process of existing visual techniques can be avoided due to the more targeted or guided techniques described herein.

[0041] The various embodiments described herein can be considered as computerized tools (e.g., any suitable combination of computer-executable hardware or computer-executable software) that can facilitate the teaching of a charged particle microscope robotic gripper via beam splitting. In various aspects, such computerized tools may include access components, overlap components, correction components, or execution components.

[0042] In various implementations, charged particle microscopy can be used. In various aspects, charged particle microscopy can demonstrate any suitable design or construction (e.g., it can be SEM, it can be TEM, it can be a dual-beam microscope). In various instances, charged particle microscopy can have any suitable vacuum chamber, and any suitable robotic gripper and mesh container can be deployed or implemented within this vacuum chamber.

[0043] In any case, it is expected that the teaching robot gripper will align its axis-axis (e.g., an axis concentric with the axis protruding from the mesh when the mesh is held by the robot gripper) with the hole-axis of the mesh container (e.g., the axis of the hole into which the axis protruding from the mesh can be inserted). As described herein, this computerized tool can accomplish such teaching by utilizing various alignment hardware that can be mounted on or within a charged particle microscope.

[0044] In particular, alignment hardware may include light sources, collimating lenses, beam splitters, reflectors, or cameras.

[0045] In various implementations, the robotic gripper and the mesh container can be attached to any suitable surface or wall within the vacuum chamber, such that the mesh container (and any mesh or mesh support that the mesh container can contain, carry, or support) is within the reach of the robotic gripper.

[0046] In all respects, the light source can be any suitable device that weighs less than one gram or otherwise within any suitable margin of one gram, and can emit or generate a beam of light of any suitable wavelength or frequency along any suitable optical axis (e.g., the light source can be an integrated circuit LED or a microlaser). In various instances, the light source can be held, held, or otherwise coupled to the robot gripper in any suitable manner such that the optical axis of the light source is collinear with the axis-axis of the robot gripper.

[0047] In various cases, a collimating lens can be any suitable lens or optical device weighing less than one gram or otherwise within any suitable margin of one gram, and can make incident light parallel or collimated (e.g., plano-convex lens, concave-convex lens, cylindrical lens, Fresnel lens). In various aspects, a collimating lens can be coupled to a light source in any suitable manner such that the collimating lens can collimate the beam emitted by the light source (e.g., can make it parallel).

[0048] In various instances, the beam splitter can be any suitable lens or optical device with a weight of less than one gram or otherwise within any suitable margin of one gram, and can split incident light into different or separate beams according to any suitable transmittance-to-reflectance ratio (e.g., dielectric beam splitter, metallic beam splitter, cubic prism beam splitter, and planar beam splitter). In particular, the beam splitter can be a cubic prism with two different beam-splitting coatings: a first beam-splitting coating on the inner diagonal plane of the cubic prism; and a second beam-splitting coating on the outer surface of the cubic prism facing the grid container. Thus, the first and second beam-splitting coatings can be considered to have a 45-degree angle between them. In other words, the first beam-splitting coating can be oriented at a 45-degree angle relative to the optical axis of the robot gripper, while the second beam-splitting coating can (alternatively) be oriented perpendicularly or orthogonally to the optical axis of the robot gripper. In various cases, the beam splitter can be coupled to a collimating lens in any suitable manner, such that the beam splitter can split the beam emitted by the light source and the beam collimated by the collimating lens.

[0049] It should be understood that in some instances, the light source, collimating lens, and beam splitter can be coupled to each other via any suitable acrylic mount, acrylic cantilever, or other acrylic mount hardware.

[0050] In various respects, the reflector can be any suitable lens or optical device capable of reflecting incident light (e.g., a mirror or a beam-splitting coating with 0% transmittance). In various instances, the reflector can be coupled to, or onto, or within, a hole in the mesh container in any suitable manner, such that the reflector is orthogonal or perpendicular to the hole-axis of the mesh container. It should be noted that the reflector cannot be held or otherwise coupled to a robotic gripper.

[0051] In various cases, the camera can be any suitable image-capturing device. In various aspects, the camera can be located above or within the vacuum chamber such that the beam split by the beam splitter can be directed toward (e.g., seen by) the camera. In some instances, the camera may be physically located inside the vacuum chamber. However, in other instances, the camera may (alternatively) be physically located outside the vacuum chamber and can gaze at or see inside the vacuum chamber through any suitable window or observation port. As noted above, it should be observed that the camera cannot be held by or otherwise coupled to the robot gripper.

[0052] Now, in various embodiments, the light source can emit a light beam. For ease of explanation, this beam will be referred to as beam A. In various aspects, beam A can pass through a collimating lens and enter a beam splitter. As mentioned above, in some cases, the beam splitter can be a cubic prism having a first beam-splitting coating along its inner diagonal plane and a second beam-splitting coating along its outer surface facing the grid container. Thus, beam A can enter the beam splitter and interact with the first beam-splitting coating.

[0053] The first beam-splitting coating allows a portion of beam A to be transmitted or passed through, while reflecting the remainder of beam A. For ease of explanation, the portion that passes or is transmitted can be referred to as beam B, and the reflected portion as beam C. In various cases, the first and second beam-splitting coatings can be oriented such that the 45-degree angle formed between them is either open or facing the camera. This allows beam C to be directed away from the camera and thus out of sight. It also allows beam B to continue propagating through the beam splitter so that it can subsequently interact with the second beam-splitting coating.

[0054] The second beam-splitting coating allows a portion of beam B to be transmitted or passed through, while reflecting the remainder of beam B. For ease of explanation, the portion that passes or is transmitted can be referred to as beam D, and the reflected portion as beam E. The orientation of the second beam-splitting coating can cause beam E to propagate back towards the first beam-splitting coating, and thereby interact with the first beam-splitting coating. In contrast, the orientation of the second beam-splitting coating can cause beam D (as an alternative) to propagate back towards the reflector within the aperture of the grid container, and thereby interact with the reflector.

[0055] As described above, the first beam-splitting coating allows a portion of the beam E to be transmitted or passed through (e.g., to propagate toward a light source), while reflecting the remainder of the beam E. For ease of explanation, the reflected portion may be referred to as beam F. In various cases, the orientation of the first beam-splitting coating can guide beam F toward the camera, and thus allow it to be seen by the camera.

[0056] Now, consider again the beam D transmitted through or by the second beam-splitting coating toward the reflector. The reflector allows beam D to be reflected back toward the second beam-splitting coating. The second beam-splitting coating allows a portion of beam D to be transmitted or passed through, while reflecting the remainder of beam D (e.g., so that it propagates back toward the reflector). For ease of explanation, the portion that passes through or is transmitted can be referred to as beam G. The orientation of the second beam-splitting coating allows beam G to propagate back toward the first beam-splitting coating, and thereby interact with the first beam-splitting coating.

[0057] As described above, the first beam-splitting coating allows a portion of the beam G to be transmitted or passed through (e.g., to propagate toward a light source), while reflecting the remainder of the beam G. For ease of explanation, the reflected portion may be referred to as beam H. In various cases, the orientation of the first beam-splitting coating can guide beam H toward the camera, and thus allow it to be seen by the camera.

[0058] Therefore, the camera can capture an image depicting two illumination spots: a first illumination spot corresponding to beam F or otherwise caused by it; and a second illumination spot corresponding to beam H or otherwise caused by it. In various instances, the image may depict only the first and second illumination spots. For example, the image may contain non-zero value pixels and zero value pixels, where some of the non-zero value pixels belong to or constitute the first illumination spot, and where the remaining non-zero value pixels belong to or constitute the second illumination spot. In various aspects, it can be determined based on this image whether the optical axis of the robot gripper is angularly aligned with the hole-axis of the mesh container.

[0059] In practice, because the first illumination spot can be caused by a beam F, because the beam F can be based on reflection caused by the second beam-splitting coating, and because the second beam-splitting coating can be orthogonal to the optical axis of the robot gripper, the image-within-the-image position of the second illumination spot can be considered to depend on or indicate the angular orientation of the optical axis. In other words, the angular orientation of the optical axis in three-dimensional space can be considered to control or influence the position of the first illumination spot within the image. In contrast, because the second illumination spot can be caused by a beam H, because the beam H can be based on reflection caused by a reflector, and because the reflector can be orthogonal to the aperture-axis of the mesh container, the image-within-the-image position of the second illumination spot can be considered to depend on the angular orientation of the aperture-axis. In other words, the angular orientation of the aperture-axis in three-dimensional space can be considered to control or influence the position of the second illumination spot within the image.

[0060] In this way, the first and second illumination spots can be seen as conveying information about whether the optical axis and aperture axis are angularly aligned. In fact, if the image shows or depicts the first and second illumination spots as concentric, the optical axis and aperture axis can be considered angularly aligned. Conversely, if the image shows or depicts the first and second illumination spots as non-concentric, the optical axis and aperture axis (as an alternative) can be considered not angularly aligned.

[0061] As described in this article, computerized tools can electronically teach a robotic gripper, based on images, how to angularly align the optical axis (and thus its axis-axis) with the hole-axis of the mesh container.

[0062] In various implementations, the access component of the computerized tool can access the image electronically. For example, the access component can electronically receive or retrieve images from a camera. In some cases, the access component can be viewed as a conduit through which other components of the computerized tool can electronically interact with the image (e.g., read, write, edit, copy, manipulate).

[0063] In various implementations, the overlapping component of the computerized tool can electronically determine whether the first illumination spot and the second illumination spot overlap each other.

[0064] Specifically, the overlapping component can electronically store, maintain, control, or otherwise access the object detection algorithm. In various aspects, the object detection algorithm can be any suitable algorithm or program for detecting or identifying how many distinct or individual objects are depicted within any given image. In some instances, the object detection algorithm can exhibit any suitable machine learning architecture. For example, the object detection algorithm can be a deep learning neural network configured to receive an input image and produce a scalar as output indicating the total number of objects depicted in the input image (from the perspective of the deep learning neural network). In this context, the deep learning neural network can have any suitable number and type of layers (e.g., an input layer, one or more hidden layers, an output layer, where any layer can be a convolutional layer, a dense layer, a long short-term memory (LSTM) layer, a non-linear layer, a pooling layer, a batch normalization layer, or a padding layer), can have any suitable number of neurons in each layer (e.g., different layers can have the same or different numbers of neurons), can have any suitable activation function in different neurons (e.g., Softmax, Sigmoid, hyperbolic tangent, linear correction unit) (e.g., different neurons can have the same or different activation functions), or can have any suitable intermediate neuron connections or inter-layer connections (e.g., forward connections, skip connections, recurrent connections). In other instances, the object detection algorithm can (alternatively) demonstrate any suitable non-machine learning architecture. For example, the object detection algorithm may include any suitable computer vision process or technique utilizing Hu moments, pixel intensity curves, oriented gradient histograms, template matching, Haar cascades, or edge boxes.

[0065] Regardless of its specific internal architecture, the object detection algorithm can be configured to determine how many distinct or individual objects are depicted in any given image. Therefore, the overlay component can electronically perform the object detection algorithm on images captured by a camera from the alignment hardware, and such execution can produce a scalar (as inferred by the object detection algorithm) indicating how many objects are depicted in the image. For example, imagine the object detection algorithm is a deep learning neural network. In this case, the overlay component can feed the image into the input layer of the object detection algorithm, which undergoes forward propagation through one or more hidden layers of the object detection algorithm, and the output layer of the object detection algorithm can compute a scalar based on the activations provided by one or more hidden layers of the object detection algorithm. As another example, imagine the object detection algorithm (as an alternative) is a non-machine learning algorithm. In this case, the overlay component can compute any Hu moments, intensity curves, or pixel histograms invoked by the object detection algorithm for the image, and the overlay component can compute a scalar based on such Hu moments, intensity curves, or pixel histograms.

[0066] In any case, the scalar can be a positive integer, and its value or size indicates how many distinct individual objects the object detection algorithm has detected or identified in the image. As mentioned above, the image can depict or show a first illumination spot and a second illumination spot. In various aspects, the image may not depict any objects other than the first and second illumination spots (although the image may depict artifacts or residuals of the first and second illumination spots, such artifacts or residuals can be removed by pixel thresholding). Therefore, there are two possible cases: a first case where the image can have two distinct individual objects, which may occur when the first and second illumination spots do not overlap each other; and a second case where the image may (alternatively) have only a single object, which (alternatively) may occur when the first and second illumination spots at least partially overlap each other.

[0067] Therefore, by utilizing object detection algorithms, the overlapping component can be considered as determining whether the first and second illumination spots overlap. If the object detection algorithm infers that the image depicts two objects, the overlapping component can determine that the first and second illumination spots in the image must not overlap or touch (e.g., otherwise, from the object detection algorithm's perspective, the first and second illumination spots would be visually indistinguishable). In contrast, if the object detection algorithm (as an alternative) infers that the image depicts one object, the overlapping component can determine that the first and second illumination spots must overlap, touch, or otherwise connect in the image.

[0068] In various implementations, the calibration components of the computerized tool can electronically identify the position or orientation of the robot gripper in three-dimensional space, which will angularly align the optical axis of the robot gripper (and thus its axis-axis) with the hole-axis of the mesh container. For ease of explanation, such identified position or orientation may be referred to as a new angular configuration. In various aspects, how the calibration components identify the new angular configuration may be based on whether the first illumination spot and the second illumination spot overlap.

[0069] Imagine an overlap component that determines the first and second illumination spots do not overlap. In this case, any pixel belonging to or constituting the first illumination spot is known, and similarly, all pixels belonging to or constituting the second illumination spot are known (e.g., because there is no overlap, and because the image may not depict any object other than the first and second illumination spots, each non-zero pixel in the image can be considered to belong to exactly one of the first and second illumination spots). Therefore, in various respects, the correction component can average the in-image positions of any pixels belonging to the first illumination spot to produce a first centroid of the first illumination spot. Similarly, the correction component can average the in-image positions of any pixels belonging to the second illumination spot to produce a second centroid of the second illumination spot. In various instances, the correction component can identify new angular configurations by applying any suitable pixel-to-physical kinematic formula to the first and second centroids. In fact, since the dimensions, relative positions, and other properties or characteristics of the robot gripper, the apertures of the mesh container, the light source, the collimating lens, the beam splitter, the reflector, and the camera are all known or otherwise controlled, any suitable mathematical formula can be derived (e.g., from classical or quantum mechanics; taking into account the angular displacement of the robot gripper, the angle of incidence or reflection of the beam interacting with the beam splitter or reflector, the transmittance to reflectance ratio of the beam splitter or reflector, the distance between the camera and the beam splitter, or the distance between the beam splitter and the reflector) that defines or quantifies how the angular movement of the robot gripper affects the in-image positions of the first and second centroids. In other words, any suitable mathematical transformation or function can exist that relates the first and second centroids to the three-dimensional position or orientation of the robot gripper. Therefore, the correction component can be considered as knowing how angular or translational deviations of the robot gripper relative to its current 3D position or orientation will cause the centroids of the first and second illumination spots to move (e.g., given the robot gripper's current in-image position, decreasing the robot gripper's pitch angle by θ degrees will move the first centroid to a first new position and simultaneously move the second centroid to a second new position; given the robot gripper's current in-image position, increasing the robot gripper's yaw angle...). This will cause the first center of mass to move to a third new position, and simultaneously cause the second center of mass to move to a fourth new position. Therefore, the correction component can utilize those mathematical transformations or functions to identify new angular configurations (e.g., to identify three-dimensional positions or orientations of the robot gripper that will cause the first and second centers of mass to become concentric or otherwise move closer to each other). It should be noted that, unlike existing visual techniques, in this case, aimless trial and error is not required to identify new angular configurations.

[0070] Now, imagine (as an alternative) that the overlap component determines that the first and second illumination spots overlap. In this case, the pixels constituting the first illumination spot may not be visually distinguishable from the pixels constituting the second illumination spot (e.g., due to the detected overlap, some pixels in the image may belong to both the first and second illumination spots, and identifying such pixels is not easy). Therefore, using pixel position averaging to locate the first and second centroids may be unreliable. In various aspects, the correction component can perform contour fitting on the image such that the contour is any suitable two-dimensional boundary (e.g., a rectangle, an ellipse) with a minimum area that defines both the first and second illumination spots (e.g., defining all non-zero pixels in the image). To achieve this, any suitable computer vision contour fitting technique can be implemented. In some cases, the correction component can implement any suitable machine learning contour fitting technique. For example, the machine learning contour fitting technique could be a deep learning neural network configured to receive the image as input and identify the vertices or edges of the contour as output. In this context, the deep learning neural network can have any suitable number and type of layers (e.g., an input layer, one or more hidden layers, and an output layer, where any layer can be a convolutional layer, a dense layer, an LSTM layer, a nonlinear layer, a pooling layer, a batch normalization layer, or a padding layer), can have any suitable number of neurons in each layer (e.g., different layers can have the same or different numbers of neurons), can have any suitable activation function in different neurons (e.g., Softmax, Sigmoid, hyperbolic tangent, linear correction unit) (e.g., different neurons can have the same or different activation functions), or can have any suitable intermediate neuron connections or inter-layer connections (e.g., forward connections, skip connections, recurrent connections). In other cases, the correction component can (alternatively) implement any suitable non-machine learning contour fitting technique. For example, non-machine learning contour techniques can include any suitable computer vision process or technique utilizing polygon approximation, convex hull computation, or curve fitting computation. In any case, the dimensions of the fitted profile are known, and the calibration component can use these dimensions to identify the first and second centroids by applying any suitable geometric formula to the fitted profile (e.g., it can be assumed that the first and second illumination spots are circles with the same radius, which can be geometrically approximated by the known dimensions of the fitted profile, and thus the first and second centroids can be located based on these radii and the known position of the fitted profile).

[0071] The inventors have now discovered that as the first and second centroids get closer to each other (e.g., as the overlap between the first and second illumination spots increases), identifying new angular configurations via pixel-to-physical kinematics formulas may become less accurate or reliable. Therefore, in various respects, the correction component can calculate or measure the distance between the first and second centroids. If this distance exceeds any suitable threshold, the correction component can determine that the first and second centroids are far enough apart that the pixel-to-physical kinematics formulas can be reliably used. Thus, as described above, the correction component can identify new angular configurations by applying these pixel-to-physical kinematics formulas to the first and second centroids. In contrast, if the distance is less than the threshold, the correction component can (alternatively) conclude that the first and second centroids are too close together to reliably use the pixel-to-physical kinematics formulas. In this case, the correction component can identify new angular configurations by having the robot gripper sweep across a series of angular configurations within any suitable threshold margin or proximity of its current angular configuration. For each of the sweep angle configurations, the correction component can calculate the corresponding area that the fitted profile has due to that sweep angle configuration. Any of these sweep angle configurations that makes the fitted profile exhibit a minimum or minimized area can be considered as bringing the first and second illumination spots as close together as possible, and therefore, such sweep angle configurations can be considered or treated as new angle configurations.

[0072] In any case, the new angular configuration can be considered as any three-dimensional position or orientation of the robotic gripper, such that the optical axis, and thus the axis-axis and hole-axis, are angularly aligned (e.g., parallel). In fact, experiments conducted by the inventors have demonstrated that the various embodiments described herein can achieve angular alignment with a resolution of less than 1 arcsecond per pixel, which can be considered highly precise. Furthermore, the various embodiments described herein achieve this angular alignment in just a few minutes. In contrast, existing visual estimation techniques often take hours or even days.

[0073] In various implementations, the execution components of the computerized tool can electronically perform or initiate any suitable electronic action based on the new angular configuration. For example, the execution components can cause the robot gripper to move to or otherwise occupy the new angular configuration. As another example, the execution components can mark or identify the new angular configuration as angularly aligned with the holes of the mesh container, making the new angular configuration easy or quick to reference or use in the future. In other words, it can now be assumed that the robot gripper knows, has learned, or has been taught to angularly align the new angular configuration with the holes of the mesh container.

[0074] While the disclosures herein to date primarily describe various embodiments in which images captured by a camera of the alignment hardware can be used to teach a robot gripper how to become angularly aligned with the holes of a mesh container, these are merely non-limiting examples. In some cases, images captured by a camera of the alignment hardware can also be used to teach a robot gripper how to become translationally aligned with a mesh container. For example, a reflector placed on or in the holes of the mesh container can be segmented, fragmented, or otherwise non-uniform. In other words, in some embodiments, the reflector may have spatially varying reflective properties. In fact, in some cases, the reflectivity of the reflector may have a maximum value at its center and thus at the center of the hole, and gradually decreases radially outward (e.g., in a continuous or stepwise manner). Therefore, since the reflector can generate a second illumination spot, the correction components can electronically determine whether the optical axis of the robot gripper is translationally aligned (e.g., collinear) with the hole-axis based on the intensity of the second illumination spot (e.g., average intensity, maximum intensity, intensity curve). If the robotic gripper is angularly aligned with the hole, and if the intensity of the second illumination spot is above any suitable threshold, it can be inferred that the optical axis of the robotic gripper points to the high reflectivity center of the reflector and is thus translationally aligned with the hole. Conversely, if the robotic gripper is angularly aligned with the hole, and if the intensity of the second illumination spot is below the threshold, it can be (alternatively) inferred that the optical axis of the robotic gripper does not point to the high reflectivity center of the reflector and is thus not translationally aligned with the hole. In this case, the correction component can cause the robotic gripper to sweep through a series of translational configurations within any suitable margin or proximity of its current translational configuration, and any sweeping translational configuration, as long as it causes the intensity of the second illumination spot to meet the threshold, can be considered to have made the robotic gripper translationally aligned with the hole.

[0075] It should be understood that the mesh container may have any suitable number of different holes, and the various embodiments described herein may be implemented for each of such different holes (e.g., to teach a robot gripper how to become angularly or translationally aligned with each of such different holes).

[0076] The various implementations described herein can be used to solve inherently technical problems (e.g., facilitating the teaching of charged particle microscope robotic grippers via beam splitting) using hardware or software. These problems are not abstract and cannot be performed as a set of human mental behaviors. Furthermore, some of the processes performed can be carried out using dedicated computers (e.g., charged particle microscopes, such as SEM, TEM, or dual-beam microscopes with vacuum chambers and mesh processing robots; optical hardware, such as cameras, light sources, beam splitters, and reflectors) to perform prescribed operations relevant to the field of charged particle microscopy.

[0077] For example, such prescribed operations may include: accessing a charged particle microscope having a vacuum chamber via a device operably coupled to a processor, wherein the vacuum chamber may include a robotic gripper and a microscope grid container; and, via this device, teaching the robotic gripper to angularly or translationally align its axis-to-axis with the aperture-to-axis of the microscope grid container based on a light source coupled to the robotic gripper. In various aspects, such prescribed operations may include: via this device causing the light source to emit a beam through a beam splitter, wherein the beam splitter is configured to direct a first portion of the beam toward a camera, wherein the beam splitter is configured to direct a second portion of the beam toward a reflector positioned on or within an aperture corresponding to the aperture-to-axis, wherein the reflector is configured to reflect the second portion of the beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the beam toward the camera. In various instances, the camera may be configured to capture an image depicting a first illumination spot caused by the first portion of the beam, and wherein the image depicts a second illumination spot caused by at least some of the second portion of the beam.

[0078] In various cases, such prescribed operations may include: determining, via the device and by applying object detection technology to the image, whether a first illumination spot and a second illumination spot overlap. In response to determining that the first illumination spot and the second illumination spot do not overlap, such prescribed operations may include: identifying a first centroid of the first illumination spot based on the average position of pixels belonging to the first illumination spot; identifying a second centroid of the second illumination spot based on the average position of pixels belonging to the second illumination spot; calculating, via the device and by one or more kinematic formulas, an angle or translational position that the robot gripper predicts will reduce the distance between the first and second centroids; and moving the robot gripper to that angle or translational position via the device.

[0079] In response to determining that the first illumination spot and the second illumination spot overlap, such prescribed operations may include: defining the first illumination spot and the second illumination spot with a minimum area profile using the device; identifying a first centroid of the first illumination spot and a second centroid of the second illumination spot using the device and via one or more geometric formulas based on the minimum area profile; in response to a distance between the first centroid and the second centroid being greater than a threshold, calculating an angle or translational position by which the robot gripper would predict a reduction in that distance using the device and via one or more kinematic formulas; and moving the robot gripper to that angle or translational position using the device.

[0080] In response to the determination that the first and second illumination spots overlap, such prescribed operations may include: defining the first and second illumination spots with a minimum area profile using the device; identifying a first centroid of the first illumination spot and a second centroid of the second illumination spot using the device and via one or more geometric formulas based on the minimum area profile; in response to a distance less than a threshold separating the first and second centroids, sweeping a series of angular or translational positions within a threshold proximity of its current angular or translational position using the device; calculating the corresponding area of ​​the minimum area profile for the series of angular or translational positions using the device; and moving the robot gripper to any of the series of angular or translational positions corresponding to the minimum area of ​​the minimum area profile using the device.

[0081] Such prescribed operations are inherently computerized. In fact, charged particle microscopes (e.g., SEM, TEM, dual-beam microscopes) are high-tech computerized devices composed of specific computerized hardware (e.g., temperature sensors, pressure sensors, voltage sensors, ion beam emitters, electron beam emitters, focusing lenses, ion detectors, electron detectors, beam apertures, fluid valves, and actuated specimen stages). Neither charged particle microscopes nor their specimen handling robotic hardware can be implemented in any reasonable or feasible way through human thought or with pen and paper without a computer. Furthermore, optical hardware that can emit, reflect, or otherwise manipulate light beams (e.g., cameras, light sources, reflectors, beam splitters) are tangible physical devices that cannot be implemented in any reasonable way through human thought without a computer.

[0082] Furthermore, the various embodiments described herein can integrate various teachings relevant to the field of charged particle microscopy into practical applications. As mentioned above, it may be desirable to teach the robotic gripper of a charged particle microscope how to align its axis-axis with the aperture-axis of the microscope's mesh container angularly or translationally. In other words, it may be desirable to identify which particular three-dimensional position or orientation of the robotic gripper aligns that axis-axis with the aperture-axis. Existing techniques facilitate such teaching or identification via pipe mirrors or cameras that attempt to visually locate this alignment. Some existing techniques involve pipe mirrors or cameras coupled to or held by a robotic gripper. Such existing techniques are imprecise (e.g., only achieving a coarse visual estimate of alignment; mixed deflections may occur because miniature cameras are typically an order of magnitude heavier than the specimen mesh) and time-consuming (e.g., iterations on the position of the robotic gripper may take hours or days). Other prior art involves pipe mirrors or cameras that are not coupled to or held by a robot gripper and have both the robot gripper and the mesh container in their field of view. These other prior art techniques require multiple cameras (e.g., top-view cameras cannot reliably determine vertical or horizontal alignment; side-view cameras cannot reliably determine horizontal or vertical alignment), remain imprecise (e.g., only a rough visual estimate of alignment can be achieved despite the lack of additional yaw), and are still time-consuming (e.g., iterations on the robot gripper's position can take hours or days, again). Therefore, the prior art suffers from various technical drawbacks.

[0083] By implementing a charged particle microscope robotic gripper taught via beam splitting, the various embodiments described herein can help improve one or more of these technical problems. Specifically, given a robotic gripper and a mesh container, the robotic gripper can be taught how to become translationally or angularly aligned with the aperture of the mesh container, based on a light source coupled to the robotic gripper. In practice, the light source emits a beam through a beam splitter, which is also coupled to the robotic gripper. The beam splitter can direct a first portion of the beam toward a camera, allowing the camera to see or capture a first illumination spot. The beam splitter can direct a second portion of the beam toward a reflector located within the aperture of the mesh container. The reflector can direct the second portion of the beam back to the beam splitter, and the beam splitter can direct at least some of this second portion toward the camera, allowing the camera to see or capture a second illumination spot. As described herein, the in-image positions of the first and second illumination spots can be considered to depend on the relative angular orientation of the robotic gripper and the aperture. If the robot gripper is angularly aligned with the hole (e.g., such that the optical axis of the light source is parallel to the hole-axis of the mesh container), this angular alignment will make the first and second illumination spots concentric. Conversely, if the robot gripper is not angularly aligned with the hole (e.g., such that the optical axis of the light source is not parallel to the hole-axis of the mesh container), this angular alignment will make the first and second illumination spots non-concentric. Therefore, the concentricity or non-concentricity of the first and second illumination spots can be considered an indicator of the angular alignment or misalignment between the robot gripper and the hole in the mesh container.

[0084] In various instances, since the dimensions, characteristics, and distances between the robot gripper, mesh container, light source, beam splitter, reflector, and camera can be known or otherwise controlled, any suitable kinematic functions or relationships can be derived that quantify how the first and second illumination spots move (e.g., two-dimensional movement in pixel space) in response to changes in the robot gripper's angular orientation (e.g., pitch or yaw in real-world physical space). Therefore, given the centroids of the first and second illumination spots, such kinematic functions or relationships can be used to identify which specific angular orientations of the robot gripper will cause these centroids to become concentric or otherwise move closer to each other. In this way, the robot gripper can be angularly aligned and taught in minutes, compared to hours or days of trial and error in existing techniques.

[0085] If the first and second illumination spots do not overlap (e.g., as determined by any suitable object detection computer vision algorithm), their centroids can be identified by averaging their respective pixel locations within the image. If the first and second illumination spots overlap, their centroids can be identified (alternatively) by defining a minimum area contour around both the first and second illumination spots and estimating the centroid using the fitted size of that contour. The inventors have found that the kinematic formula works well when the centroids of the first and second illumination spots are far apart, and poorly when they are closer together. Therefore, if the centroids identified in this contour-based manner are less than a threshold distance apart, angular alignment cannot be achieved via the kinematic formula. Instead, the robotic gripper can sweep across a series of adjacent angular orientations, and any orientation that minimizes the area of ​​the fitted contour can be considered to achieve angular alignment.

[0086] In some cases, such illumination spots can even be used to teach a robotic gripper how to become translationally aligned with the aperture of a mesh container. In practice, the reflectivity of a reflector may vary in space, such that the center of the reflector, and consequently the center of the aperture, is associated with a maximum or otherwise unique reflectivity. Therefore, if the intensity of the second illumination spot corresponds to this maximum or otherwise unique reflectivity value (e.g., matches, is commensurate), the robotic gripper can be considered translationally aligned with the aperture. Conversely, if the intensity of the second illumination spot does not correspond to this maximum or otherwise unique reflectivity value (e.g., does not match, is asymmetrical), the robotic gripper can be considered not translationally aligned with the aperture. In this case, the robotic gripper can sweep across a series of adjacent translational positions, and any position where the intensity of the second illumination spot corresponds to the maximum or otherwise unique reflectivity value of the reflector can be considered as achieving translational alignment.

[0087] In this way, the various embodiments described herein can be considered innovative technologies that efficiently and effectively teach a robotic gripper how to angularly or translationally align with the holes of a mesh container. Various experiments conducted by the inventors have confirmed that such embodiments can achieve angular alignment in just a few minutes, with a resolution of less than one arcsecond per pixel. This can be considered a higher or better resolution than that achievable with existing visual inspection techniques (even after iterative iterations that take hours or days). Furthermore, these experiments involve a light source, collimating lens, and beam splitter coupled together by a mounting bracket, all with a total weight of less than 2 grams. This demonstrates that the various embodiments described herein can eliminate the excessive deflection problem that hinders some existing visual inspection techniques. At least for the reasons stated above, the various embodiments described herein can be considered to resolve or mitigate various problems or drawbacks that conflict with existing technologies. Therefore, the various embodiments described herein can be considered concrete and tangible technical improvements in the field of charged particle microscopy. Thus, the various embodiments described herein are certainly qualified as useful and practical applications of computers.

[0088] Furthermore, it should be understood that the various implementations described herein can control real-world tangible devices. In fact, various implementations may involve adjusting the angle or translational position of a real-world robotic gripper deployed within a real-world charged particle microscope.

[0089] Figure 1 An exemplary non-limiting block diagram of a scientific instrument module 102 according to various embodiments described herein is shown.

[0090] In various implementations, the scientific instrument module 102 may be implemented by circuitry such as a programmed computing device (e.g., including electrical or optical components). The logic components of the scientific instrument module 102 may be included in a single computing device, or may be distributed across multiple computing devices communicating with each other, as appropriate. (References herein) Figure 38 and Figure 40 Examples of computing devices that can implement the scientific instrument module 102 individually or in combination are discussed, and references are made to... Figure 39 and Figure 41 Examples of systems or networks that can interconnect computing devices in scientific instrument module 102 across one or more computing devices are discussed.

[0091] Scientific instrument module 102 may include a first logic element 104 and a second logic element 106. As used herein, the term "logic element" may include means for performing a set of operations associated with that logic element. For example, any logic element included in scientific instrument module 102 may be implemented by one or more computing devices programmed with instructions to cause one or more processing devices of the computing device to perform an associated set of operations. In a particular embodiment, a logic element may include one or more non-transitory computer-readable media having instructions on them that, when executed by one or more processing devices of the one or more computing devices, cause the one or more computing devices to perform an associated set of operations. As used herein, the term "module" may refer to a collection of one or more logic elements that together perform the functionality associated with the module. Different logic elements in a module may take the same form or may take different forms. For example, some logic elements in a module may be implemented by a programmed general-purpose processing device, while other logic elements in the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may omit one or more logic elements described in the associated figures; for example, when the module is to perform a subset of the operations discussed herein with reference to the module, the module may include a subset of the logic elements depicted in the associated figures.

[0092] In various embodiments, a scientific instrument corresponding to scientific instrument module 102 may be present. In various aspects, the scientific instrument can be any suitable computerized device capable of electronically measuring scientifically relevant, clinically relevant, or research-related characteristics, properties, or attributes of an analytical specimen (e.g., a known or unknown mixture, compound, or collection of substances). As a non-limiting example, the scientific instrument may be a scanning electron microscope. In this case, the scientific instrument can capture images of the analytical specimen to measure or determine the surface morphology, surface material composition, or crystal structure of the analytical specimen. Another non-limiting example is that the scientific instrument may be a transmission electron microscope. In this case, the scientific instrument can capture images of the interior of the analytical specimen to measure or determine the details of the internal wall structure of the analytical specimen. As another non-limiting example, the scientific instrument could even be a dual-beam microscope. In this case, the scientific instrument can capture images of the analytical specimen in addition to grinding it. As a more general non-limiting example, the scientific instrument may be any suitable type of charged particle microscope (e.g., some types of microscopes can use non-electron ion beams to capture images). In various instances, scientific instruments may contain or otherwise have a vacuum chamber in which robotic grippers and mesh containers may be deployed.

[0093] In various implementations, the first logic component 104 may involve accessing scientific instruments. Therefore, electronic instructions or commands can be transmitted to or from the scientific instruments.

[0094] In various embodiments, the second logic component 106 may relate to how the robotic gripper, teaching a scientific instrument, angularly or translationally aligns its axis-to-axis with the aperture-to-axis of the mesh container, based on a light source coupled to the robotic gripper. More specifically, the light source may emit a beam of light through a beamsplitter, which is also coupled to the robotic gripper. The beamsplitter may direct a first portion of the beam toward a camera associated with a vacuum chamber, and may also direct a second portion of the beam toward a reflector positioned within the aperture of the mesh container. The reflector may reflect the second portion of the beam back to the beamsplitter, and the beamsplitter may direct at least some of the second portion of the beam toward the camera. Thus, the camera may capture an image depicting two illumination spots: a first illumination spot caused by the first portion of the beam; and a second illumination spot caused by at least some of the second portion of the beam. In various aspects, whether the first and second illumination spots are concentric may be considered as an indication of whether the axis-to-axis of the robotic gripper is angularly aligned with the aperture-to-axis of the mesh container. In various instances, given the current centroids of the first and second illumination spots, the second logic component 106 may involve determining, via any suitable pixel-to-physical kinematics transformation, the angular position of the robot gripper that would cause the first and second centroids to become concentric or otherwise move closer to each other. Thus, the robot gripper can be taught to become angularly aligned with the mesh container without visual error testing. If the first and second spots do not overlap, the second logic component 106 can identify the centroids by averaging the pixel positions of the first and second illumination spots, respectively. If the first and second spots overlap, the second logic component 106 can (alternatively) identify the centroids by fitting a contour around the first and second illumination spots and geometrically estimating the centroids based on the fitted dimensions of the contours. In some cases, if the centroid spacing is less than a threshold distance, the second logic component 106 can avoid using pixel-to-physical kinematics transformations to identify the angular position of the robot gripper that would cause the first and second centroids to become concentric or otherwise move closer to each other. Conversely, if the centroid spacing is less than a threshold distance, the second logic unit 106 can make the robot gripper sweep a series of angular positions within a threshold proximity of its current angular position, and any sweep angular position can be considered to achieve angular alignment as long as it minimizes the area of ​​the contour.

[0095] Therefore, the scientific instrument module 102 can facilitate the teaching of a charged particle microscope robotic gripper via beam splitting.

[0096] Figure 2 This is an exemplary non-limiting flowchart of a computer-implemented method 200 according to various embodiments described herein. The operation of the computer-implemented method 200 can be used in any suitable environment to perform any suitable operation (e.g., it can be performed by...). Figure 1 , Figure 37 , Figure 38 , Figure 39 , Figure 40 and Figure 41 (This refers to the execution or use in conjunction with any of the various modules, computing devices, or graphical user interfaces described). Figure 2 In this context, operations are shown once in a specific order, but they can be reordered or repeated as needed and as appropriate (e.g., different operations can be executed in parallel where appropriate).

[0097] In various aspects, action 202 may include performing a first operation to access a charged particle microscope having a vacuum chamber via a device operatively coupled to the processor, wherein the vacuum chamber may include a robotic gripper and a microscope mesh container. In various cases, the first logic component 104 may perform or otherwise facilitate action 202.

[0098] In various aspects, action 204 may include performing a second operation by means of the device, based on a light source coupled to the robot gripper, teaching the robot gripper to align its axis-to-axis with the aperture-to-axis of the microscope mesh container angularly or translationally. In various instances, second logic component 106 may perform or otherwise facilitate action 204.

[0099] Therefore, the computer-implemented method 200 can facilitate the teaching of a charged particle microscope robotic gripper via beam splitting.

[0100] Figure 3 A block diagram of an exemplary non-limiting system according to one or more embodiments described herein is shown, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0101] In various embodiments, a charged particle microscope 302 may be used. In various aspects, the charged particle microscope 302 may be as described above. That is, the charged particle microscope 302 may be any suitable computerized device that can electronically capture suitable images of suitable analytical specimens using its constituent hardware (e.g., electron source, anode, condenser, condenser aperture, scanning coil, objective lens, objective aperture, deflector, condenser lens, astigmatism corrector, electron detector, X-ray detector, driveable specimen stage). As a non-limiting example, the charged particle microscope 302 may be any suitable SEM. As another non-limiting example, the charged particle microscope 302 may be any suitable TEM. As yet another non-limiting example, the charged particle microscope 302 may be any suitable dual-beam microscope.

[0102] Although not explicitly shown in the figure, the charged particle microscope 302 can be electronically integrated with any suitable human-machine interface device that can be connected to the charged particle microscope 302 remotely or locally. Therefore, users or technicians associated with the charged particle microscope 302 can interact with or otherwise control it. Some non-limiting examples of the human-machine interface device could be a keyboard, a keypad, a touchscreen, or a voice command system for the charged particle microscope 302.

[0103] In any case, the charged particle microscope 302 may include a vacuum chamber 304. In various aspects, the vacuum chamber 304 may be any suitable type of structural shell or enclosure of any suitable size or shape, made of any suitable material (e.g., stainless steel, titanium), and have an internal volume that can be evacuated or expelled by any suitable vacuum pump to achieve any suitable vacuum pressure. In practice, when the charged particle microscope 302 is expected to scan any given specimen, such a scan can be performed under a vacuum within the vacuum chamber 304. In short, if such a scan is not performed under a vacuum, any beam of charged particles used by the charged particle microscope 302 for such a scan will become unstable or interfered with due to collisions with gas molecules.

[0104] In various implementations, the microscope grid container 306 and the robotic gripper 308 may be implemented or deployed within the vacuum chamber 304.

[0105] In various instances, the microscope grid container 306 can be any suitable structure of any suitable size or shape, made of any suitable material (e.g., stainless steel, titanium, aluminum), and configured to physically hold or support any suitable number of microscope grids. In various cases, the microscope grid can be any suitable divided or undivided plate, petri dish, or specimen plate on which any suitable specimen can be placed or arranged for carrying, transporting, or transferring. Non-limiting examples of microscope grids may include: copper grids; gold grids; quantifoil grids; carbon grids; silicon nitride grids; mesh-like grids; porous grids; or support membranes.

[0106] In various respects, the robotic gripper 308 can be any suitable automated end effector capable of controllable movement through three-dimensional space. Specifically, the robotic gripper 308 can be any suitable autohand, autogripper, autoclaw, autohook, or autosuction cup attached to any suitable articulated or telescopic arm having any suitable number and type of kinematic degrees of freedom. In various cases, the robotic gripper 308 can be constructed from any suitable type of linear or rotary actuator, such as: electric, electronic, or piezoelectric linear or rotary actuators (e.g., servo motors); pneumatic linear or rotary actuators; or hydraulic linear or rotary actuators. Prior to microscopic scanning, the robotic gripper 308 can physically grasp a given microscope grid from the microscope grid container 306, transport the given microscope grid angularly or translationally to an actuable stage (not shown) within the vacuum chamber 304, and then release its grip, thereby placing or arranging the given microscope grid onto the actuable stage in preparation for microscopic scanning. Conversely, after a microscope scan, the robotic gripper 308 can physically grasp a given microscope grid container from the surface of an actuable stage, transport the given microscope grid back to the microscope grid container 306 at an angle or by translation, and then release the grip to place or rearrange the given microscope grid onto or into the microscope grid container 306.

[0107] Specifically, the microscope grid (or any suitable auxiliary hardware that can be coupled to the microscope grid) may include protruding shafts. In various aspects, such shafts may be configured to mate with corresponding holes in the microscope grid container 306. In other words, such shafts are capable of being inserted into holes in the microscope grid container 306. When the shaft is inserted into the hole in the microscope grid container 306, the microscope grid can be considered to be slidably or detachably coupled to the microscope grid container 306. Thus, in some cases, the microscope grid container 306 can be considered as a structural frame, support, or other fixing device to which the microscope grid can be hooked or suspended via such shaft-to-hole insertion.

[0108] To avoid damaging the charged particle microscope 302, and to avoid damaging or contaminating any specimens that will be scanned by the charged particle microscope 302, it may be desirable for the teaching robot gripper 308 to be angularly or translationally aligned with the aperture of the microscope grid container 306, with the axis protruding from any given microscope grid. As described herein, system 322 can facilitate such teaching by utilizing various alignment hardware 310.

[0109] In various implementations, alignment hardware 310 may include a light source 312, a collimating lens 314, a beam splitter 316, a reflector 318, or a camera 320.

[0110] In various instances, the light source 312 can be any suitable optical device with a mass of less than 1 gram or otherwise within any suitable threshold margin of 1 gram, and capable of emitting a focused beam of any suitable wavelength. As a non-limiting example, the light source 312 can be any suitable microspot-size LED manufactured or produced on a printed circuit board of a few square millimeters or centimeters. As another non-limiting example, the light source 312 can be a miniature laser.

[0111] In various examples, collimating lens 314 can be any suitable optical device with a mass of less than 1 gram or otherwise within any suitable threshold margin of 1 gram, and capable of making incident rays parallel or collimated to each other. As a non-limiting example, collimating lens 314 can be any suitable plano-convex lens. As a non-limiting example, collimating lens 314 can be any suitable biconvex lens. As a non-limiting example, collimating lens 314 can be any suitable concave-convex lens. As yet another non-limiting example, collimating lens 314 can be any suitable aspherical lens. As yet another non-limiting example, collimating lens 314 can be any suitable cylindrical lens. As yet another non-limiting example, collimating lens 314 can be any suitable Fresnel lens.

[0112] In various cases, beam splitter 316 can be any suitable optical device with a mass of less than 1 gram or otherwise within any suitable threshold margin of 1 gram, capable of transmitting a portion of the incident light while reflecting the remainder. As a non-limiting example, beam splitter 316 can be a cubic prism beam splitter. As another non-limiting example, beam splitter 316 can be a planar beam splitter. It should be understood that beam splitter 316 can utilize or implement any suitable type of beam-splitting coating. As a non-limiting example, beam splitter 316 can utilize any suitable dielectric beam-splitting coating. As a non-limiting example, beam splitter 316 can utilize any suitable metallic beam-splitting coating.

[0113] In all respects, reflector 318 can be any suitable optical device capable of reflecting incident light. As a non-limiting example, reflector 318 can be any suitable planar dielectric mirror. As another non-limiting example, reflector 318 can be any suitable planar metallic mirror. As yet another non-limiting example, reflector 318 can be any suitable planar acrylic mirror. It should be understood that in some cases, reflector 318 may have non-zero transmittance.

[0114] In various instances, camera 320 can be any suitable optical device capable of capturing or generating an image in response to incident light. As a non-limiting example, camera 320 can be any suitable charge-coupled device (CCD) camera. As another non-limiting example, camera 320 can be any suitable complementary metal-oxide-semiconductor (CMOS) camera. In various cases, camera 320 can be configured to focus at infinity (e.g., it can be configured to a focal length close to or approximately infinity). Such infinity focusing can be considered as making camera 320 more sensitive to changes in the angle of incident light than to changes in the translation of the incident light. In effect, the image plane of the camera can be considered to be located at the focal length of the camera. When the focal length of the camera lens is set to infinity, if two translationally offset beams passing through the camera lens are parallel to each other before passing through the camera lens, then these two translationally offset beams can be considered to be projected onto a coincident point on the image plane. Now, if those translationally offset beams (as an alternative) are not parallel to each other before passing through the camera lens, then they may (as an alternative) be projected onto different or separate points on the image plane. Furthermore, those different or individual points can: change or alter with the angle of incidence between those non-parallel, translationally offset beams; and will not change or alter with changes in translational offset between the two beams. Therefore, by setting camera 320 to focus at infinity, camera 320 can be considered sensitive to changes in the angle of incidence of the incident light, but insensitive to such translational changes of the incident light.

[0115] In various embodiments, the robotic gripper 308 and the microscope grid container 306 may be mated or equipped with alignment hardware 310. In practice, in various aspects, the light source 312, collimating lens 314, and beam splitter 316 may be coupled to or otherwise held by the robotic gripper 308. In various instances, the reflector 318 may be located within or above the aperture of the microscope grid container 306. In various cases, the camera 320 may be located within or near the vacuum chamber 304 to be able to see or capture light reflected from the beam splitter 316. Regarding... Figures 4 to 12 The non-restrictive aspects are described.

[0116] Figures 4 to 12 Exemplary non-limiting block diagrams are shown according to one or more embodiments described herein, illustrating how the robotic gripper 308 and the microscope mesh container 306 can be mated or equipped with alignment hardware 310. It should be understood that... Figures 4 to 12 It may not be drawn to scale.

[0117] First, consider Figure 4In various embodiments, the microscope mesh container 306 and the robotic gripper 308 can be attached to any suitable inner wall or surface of the vacuum chamber 304 in any suitable manner (e.g., via bolts, screws, or any other suitable mechanical fasteners) such that the microscope mesh container 306 is within the reach of the robotic gripper 308. It should be understood that, for ease of illustration, Figure 4 Only a conceptual description of the robotic gripper 308 and the microscope mesh container 306 is shown. In reality, although... Figure 4 Roughly represented as a three-jointed articulated arm, the robot gripper 308 can exhibit any suitable level of mechanical complexity and, as mentioned above, can have any suitable kinematic degrees of freedom. In fact, the robot gripper 308 is capable of operating in three translational directions (e.g., Figure 4 The X, Y, and Z directions) and three angular directions (e.g., pitch, which can be considered as circumference) and the three angular directions (e.g., pitch direction, which can be considered as circumference). Figure 4 The X-direction is the direction of pivoting or rotation; the yaw direction can be considered as circumference. Figure 4 The Z-direction is the direction of pivoting or rotation; and the rolling direction can be considered as around... Figure 4 The Y-direction pivoting or rotating direction moves its end effector. Similarly, although Figure 4 While the microscope grid container 306 may be roughly represented as a simple block of straight lines, it should be understood that the microscope grid container 306 can be any suitable static or stationary structure (or, in some cases, even a dynamic or movable structure), with any shape or size, and exhibiting any suitable level of structural complexity.

[0118] In any case, the robotic gripper 308 can be considered to have an axis-axis line 402. In various respects, if the robotic gripper 308 holds the microscope grid, the axis-axis line 402 can be any linear axis that extends longitudinally through the center of an axis protruding from the microscope grid. In practice, the robotic gripper 308 can be configured or designed to hold the microscope grid in a uniform or otherwise repeatable manner, and due to such uniform or repeatable gripping, any direction in which the axis of those microscope grids extends can be referred to as the axis-axis line 402.

[0119] In various aspects, the microscope grid container 306 may include a hole 404. In various instances, the hole 404 may be configured or sized to receive an axis protruding from the microscope grid. In various cases, the hole 404 may be considered to have a hole-axis line 406. In various aspects, the hole-axis line 406 may be any linear axis extending longitudinally through the center of the hole 404.

[0120] In various instances, how the teachable robot gripper 308 becomes angularly or translationally aligned with the hole 404 of the microscope mesh container 306 can be considered equivalent to identifying which particular angular or translational position of the robot gripper 308 would cause the axis-axis 402 to become parallel or collinear with the hole-axis 406.

[0121] Now, in various cases, the light source 312, collimating lens 314, and beam splitter 316 can all be coupled to or otherwise held by the robotic gripper 308 (e.g., via any suitable mechanical fastener). As a non-limiting example, any suitable mounting bracket (not shown) may be present to which the light source 312, collimating lens 314, and beam splitter 316 can be mechanically attached or mounted, and the robotic gripper 308 can grasp, hold, or otherwise hold the mounting bracket. It should be noted that since the weight of the light source 312, collimating lens 314, and beam splitter 316 (and the mounting bracket, if applicable) can each be less than one gram or otherwise within any suitable margin of one gram, they can be collectively considered not to cause the robotic gripper 308 to exhibit a significantly greater positional deflection than the positional deflection that occurs when the robotic gripper 308 holds the microscope grid.

[0122] In any case, the light source 312 and the collimating lens 314 can be coupled to or held by the robot gripper 308, such that the axis-axis 402 can be regarded as the optical axis of the light source 312. In other words, the light beam emitted by the light source 312 and collimated by the collimating lens 314 can be regarded as traveling along or parallel to the axis-axis 402.

[0123] In various respects, the beam splitter 316 may be coupled to or held by the robot gripper 308, such that the beam splitter 316 can receive a beam of light emitted by the light source 312 and collimated by the collimating lens 314. In other words, the collimating lens 314 may be located between the light source 312 and the beam splitter 316.

[0124] As shown in the figures, in various examples, beam splitter 316 can be a cubic prism with two different beam-splitting coatings: beam-splitting coating 408; and beam-splitting coating 410. In various cases, beam-splitting coating 408 can exhibit any suitable material composition (e.g., medium, metal) and any suitable reflectance to transmittance ratio (e.g., 50 / 50 ratio; 60 / 40 ratio). Similarly, beam-splitting coating 410 can exhibit any suitable material composition and any suitable reflectance to transmittance ratio, which may be the same as or different in size or ratio from beam-splitting coating 408. In various aspects, as shown, beam-splitting coating 408 can be disposed, positioned, or otherwise implemented on or along the inner diagonal plane of beam splitter 316. In contrast, and as shown, beam-splitting coating 410 can be disposed, positioned, or otherwise implemented on or along any outer surface of beam splitter 316 facing microscope grid container 306. Therefore, the beam-splitting coating 410 can be considered as orthogonal or perpendicular to the axis-axis 402, and the beam-splitting coating 408 can be considered as being at a 45-degree angle to the beam-splitting coating 410. Although not explicitly shown, any other outer surface of the beam splitter 316 may have any suitable anti-reflective coating.

[0125] In various respects, as shown in the figure, reflector 318 can be coupled (e.g., via any suitable mechanical fastener) to the interior of bore 404 such that reflector 318 can be considered perpendicular or orthogonal to bore-axis 406. In fact, in Figure 4 In a non-limiting example, reflector 318 may be considered as attached to the bottom surface of aperture 404. However, this is merely a non-limiting example. In other embodiments, reflector 318 may (alternatively) be mechanically positioned or suspended at any other depth within aperture 404, or even mechanically positioned or suspended outside aperture 404 such that reflector 318 is perpendicular or orthogonal to aperture-axis 406. In some instances, reflector 318 may be considered as a temporary fixture above or within aperture 404 (e.g., reflector 318 can be removed from aperture 404 when it is desired to use or operate charged particle microscope 302 to perform an actual scan of the specimen). However, in other instances, reflector 318 may be considered as a permanent fixture above or within aperture 404 (e.g., reflector 318 cannot be removed from aperture 404, such that reflector 318 can remain inside aperture 404 even when it is desired to use or operate charged particle microscope 302 to perform an actual scan of the specimen).

[0126] In various cases, as shown, camera 320 may be coupled (e.g., via any suitable mechanical fastener) to any suitable inner wall or surface of vacuum chamber 304 such that camera 320 faces beam splitter 316. However, this is merely a non-limiting example. In other embodiments, camera 320 may (alternatively) be located or positioned outside vacuum chamber 304, though still facing beam splitter 316 due to any suitable window or observation port (not shown) of vacuum chamber 304. In some aspects, the 45-degree angle formed between beam splitting coating 408 and beam splitting coating 410 may be considered as facing or extending toward camera 320.

[0127] Now, consider Figure 5 In various embodiments, as shown, the light source 312 can emit a light beam 502. The light beam 502 can pass through a collimating lens 314, allowing it to travel along an axis-axis 402. In various examples, the light beam 502 can exit from the collimating lens 314 and correspondingly enter a beam splitter 316. As it propagates through the beam splitter 316, the light beam 502 can reach and interact with a beam-splitting coating 408.

[0128] consider Figure 6 In various respects, the beam-splitting coating 408 can (assuming lossless or zero absorption conditions) allow a portion of the beam 502 to pass through (e.g., according to the transmittance value or characteristics of the beam-splitting coating 408), and can allow the remaining portion of the beam 502 to be reflected (e.g., according to the reflectance value or characteristics of the beam-splitting coating 408). The portion of the beam 502 that passes through the beam-splitting coating 408 may be referred to as beam 602, while the portion of the beam 502 that is reflected by the beam-splitting coating 408 may be referred to as beam 604. Given that the beam-splitting coating 408 is oriented at a 45-degree angle relative to the axis-axis 402, beam 604 can be considered to travel or propagate perpendicular to or orthogonal to the axis-axis 402 in a direction away from the camera 320. In contrast, beam 602 can be considered to travel or propagate along the axis-axis 402 in the same direction as beam 502. As it propagates through the beam splitter 316, beam 602 can reach the beam-splitting coating 410 and thereby interact with it.

[0129] consider Figure 7 In various respects, the beam-splitting coating 410 can (assuming lossless or zero absorption conditions) allow a portion of the beam 602 to pass through (e.g., depending on the transmittance value or characteristics of the beam-splitting coating 410), and can allow the remainder of the beam 602 to be reflected (e.g., depending on the reflectance value or characteristics of the beam-splitting coating 410). The portion of the beam 602 that passes through the beam-splitting coating 410 may be referred to as beam 902 (up to...). Figure 9(This will be discussed later), and the portion of beam 602 reflected by the beam-splitting coating 410 (as an alternative) can be referred to as beam 702. Given the perpendicular or orthogonal orientation of the beam-splitting coating 410 relative to the axis-axis 402, beam 702 can be considered to travel or propagate along the axis-axis 402 in the direction opposite to that of beam 602. Therefore, as it propagates through the beam splitter 316, beam 702 can reach the beam-splitting coating 408 and thereby interact with it.

[0130] consider Figure 8 In various respects, the beam-splitting coating 408 allows some of the beam 702 to pass through and allows the remaining portion of the beam 702 to be reflected (alternatively). The portion of the beam 702 that passes through the beam-splitting coating 408 may be referred to as beam 802, while the portion of the beam 702 that is reflected (alternatively) by the beam-splitting coating 408 may be referred to as beam 804. Given that the beam-splitting coating 408 is oriented at a 45-degree angle relative to the axis-axis 402, beam 802 can be considered to travel or propagate along the axis-axis 402 toward the light source 312. In contrast, beam 804 can be considered to travel or propagate perpendicular to or orthogonal to the axis-axis 402 toward the camera 320. Therefore, the camera 320 can be considered to see, capture, or otherwise observe beam 804.

[0131] Now, consider Figure 9 As described above, beam 902 can be any portion of beam 602 passing through beam-splitting coating 410. Given the perpendicular or orthogonal orientation of beam-splitting coating 410 relative to axis-axis 402, beam 902 can be considered to travel or propagate along axis-axis 402 in the same direction as beam 602. It should be noted that because beam-splitting coating 410 can be applied to the outer surface of beam splitter 316 facing aperture 404 of microscope grid container 306, beam 902 can be considered to propagate toward aperture 404 of microscope grid container 306. Therefore, during propagation, beam 902 can reach reflector 318 and thereby interact with it. It should be noted that if robot gripper 308 and aperture 404 are not yet aligned, beam 902 can be considered not propagating along or parallel to aperture-axis 406 (e.g., axis-axis 402 may not yet be parallel to...). Figure 9 (The Y-axis). However, for the sake of illustration, this lack of parallel propagation is not shown.

[0132] consider Figure 10In all respects, reflector 318 can (assuming lossless, zero absorption, and zero transmittance conditions) reflect beam 902. Therefore, beam 902 can now propagate back toward beam splitter 316. It should be noted that if the robot gripper 308 and aperture 404 are not aligned, beam 902 can be considered to have propagated without following or paralleling the aperture-axis 406 and axis-axis 402 after such reflection. However, for ease of illustration, this lack of parallel propagation is not shown. In any case, during propagation, beam 902 can reach and thus interact with the beam-splitting coating 410.

[0133] consider Figure 11 In various respects, the beam-splitting coating 410 allows some of the beam 902 to pass through and allows the remainder of the beam 902 to be reflected (e.g., reflected back toward aperture 404). The portion of the beam 902 that passes through the beam-splitting coating 410 may be referred to as beam 1102. It should be noted that because beam 902 may not propagate along or orthogonal to axis-axis 402, beam 1102 may also not propagate along or parallel to axis-axis 402. In any case, as it propagates through beam splitter 316, beam 1102 may reach beam-splitting coating 408 and thereby interact with it.

[0134] consider Figure 12 In various respects, the beam-splitting coating 408 allows some of the beam 1102 to pass through and allows the remaining portion of the beam 1102 to be reflected (alternatively). The portion of the beam 1102 that passes through the beam-splitting coating 408 may be referred to as beam 1202, while the portion of the beam 1102 that is reflected by the beam-splitting coating 408 may be referred to as beam 1204. Given that the beam-splitting coating 408 is oriented at a 45-degree angle relative to the axis-axis 402, beam 1202 can be considered to travel or propagate toward the light source 312 (e.g., although beam 1202 may not be along or parallel to the axis-axis 402). In contrast, beam 1204 can be considered to travel or propagate toward the camera 320 (e.g., although beam 1204 may not be perpendicular or orthogonal to the axis-axis 402). Therefore, the camera 320 can be considered to see, capture, or otherwise observe beam 1204.

[0135] Return to reference Figure 3 System 322 can be electronically integrated with charged particle microscope 302 or alignment hardware 310 via any suitable wired or wireless electronic connection. In various cases, system 322 can electronically teach robotic gripper 308 how to align axis-axis line 402 with hole-axis line 406 by utilizing information captured by camera 320.

[0136] In various aspects, system 322 may include processor 324 (e.g., computer processing unit, microprocessor) and non-transitory computer-readable storage 326 operably connected to or coupled to processor 324, or operably or communicatively connected to processor 324. Non-transitory computer-readable storage 326 may store computer-executable instructions that, when executed by processor 324, cause processor 324 or other components of system 322 (e.g., access component 328, overlap component 330, correction component 332, execution component 334) to perform one or more actions. In various embodiments, non-transitory computer-readable storage 326 may store computer-executable components (e.g., access component 328, overlap component 330, correction component 332, execution component 334), and processor 324 may execute computer-executable components.

[0137] In various embodiments, system 322 may include access component 328. In various aspects, access component 328 may electronically access charged particle microscope 302 or alignment hardware 310. That is, access component 328 may electronically communicate or otherwise electronically interact with charged particle microscope 302 (e.g., with robotic gripper 308) or alignment hardware 310 (e.g., with light source 312 or with camera 320) (e.g., transmitting electronic instructions or commands to it, receiving electronic data from it). Therefore, access component 328 can be considered as an agent or channel for other components of system 322 to interact, communicate, or otherwise operate the charged particle microscope or alignment hardware 302 or 310. In various instances, as described herein, access component 328 may access images captured by camera 320. In various cases, this image may depict a first illumination spot corresponding to beam 804 and a second illumination spot corresponding to beam 1204.

[0138] In various embodiments, system 322 may include an overlap component 330. In various aspects, as described herein, the overlap component 330 may determine whether a first illumination spot and a second illumination spot in the image overlap each other via any suitable object detection algorithm.

[0139] In various embodiments, system 322 may include a correction component 332. In various instances, as described herein, the correction component 332 may identify the angular position of the robot gripper 308 that would angularly align the axis-axis 402 with the hole-axis 406 based on whether the first illumination spot and the second illumination spot overlap.

[0140] In various embodiments, system 322 may include execution component 334. In various cases, as described herein, execution component 334 may facilitate any suitable electronic action based on the identified angular position.

[0141] It should be noted that in various instances, access component 328, overlap component 330, correction component 332, and execution component 334 can be collectively considered as one or more software components 327 of system 322. In all aspects, it should be understood that, for ease of explanation and illustration, this document primarily describes one or more software components 327 as comprising four components (e.g., access component 328, overlap component 330, correction component 332, and execution component 334). However, one or more software components 327 are not limited to being implemented as exactly four components in every embodiment. In fact, in some embodiments, the functionality of these four components described herein can be combined in any suitable manner so that these functions are implemented in four components or by fewer than four components (e.g., in some cases, a single component can perform all of these functions described herein related to access component 328, overlap component 330, correction component 332, and execution component 334). In other embodiments, the functionality of the four components described herein may (as an alternative) be distributed, separated, split, or fragmented in any suitable manner so that these functions are implemented in or by four or more components (e.g., two or more components may facilitate functionality that can be performed by access component 328; two or more components may facilitate functionality that can be performed by overlapping component 330; two or more components may facilitate functionality that can be performed by correction component 332; two or more components may facilitate functionality that can be performed by execution component 334).

[0142] Figure 13 A block diagram of an exemplary non-limiting system including an image is shown according to one or more embodiments described herein, which facilitates teaching a charged particle microscope robotic gripper via beam splitting.

[0143] In various embodiments, access component 328 can access image 1302 electronically. In some aspects, access component 328 can electronically retrieve, electronically receive, or otherwise electronically acquire image 1302 from camera 320. In various instances, image 1302 can exhibit any suitable format, size, or dimension. As a non-limiting example, image 1302 can be an x-by-y pixel array, where x and y are any suitable positive integers. In various cases, image 1302 can describe or illustrate illumination spots 1304 and 1306. In various aspects, illumination spot 1304 can be any suitable circular cluster of pixels in image 1302 having a non-zero intensity value caused by beam 804. In other words, as described above, beam 804 can be directed by beam splitter 316 toward camera 320 for projection onto the viewfinder lens of camera 320, and illumination spot 1304 can be considered as the resulting pixel value caused by such lens projection. Similarly, in various instances, the illumination spot 1306 can be any suitable circular cluster of pixels in the image 1302, having a non-zero intensity value caused by the beam 1204. In other words, as described above, the beam 1204 can be directed by the beam splitter 316 towards the camera 320 for projection onto the viewfinder of the camera 320, and the illumination spot 1306 can be considered as the resulting pixel value caused by such lens projection. In some cases, the illumination spot 1304 and the illumination spot 1306 can have the same or different sizes or radii. In some instances, the illumination spot 1304 and the illumination spot 1306 can have the same or different pixel intensities.

[0144] In all respects, image 1302 can be considered as depicting or showing only illumination spots 1304 and 1306. In other words, image 1302 can be considered as having pixels with non-zero intensity values ​​and pixels with zero intensity values, some of which pixels with non-zero intensity values ​​may belong to or constitute illumination spot 1304, and the remaining pixels with non-zero intensity values ​​may belong to or constitute illumination spot 1306.

[0145] In various respects, the illumination spot 1304 can be considered as being generated or caused by the reflective interaction of the beam-splitting coating 410. In fact, the illumination spot 1304 can be considered as a visual representation of the beam 804 captured by the camera 320, and the beam 804 can be considered as originating from the beam 702 reflected by the beam-splitting coating 410. Since the illumination spot 1304 can be generated or caused by the reflective interaction of the beam-splitting coating 410, and since the beam-splitting coating 410 can be considered as always being perpendicular or orthogonal to the axis-axis 402, the illumination spot 1304 can be considered as serving as a visual indication of the axis-axis 402 or otherwise corresponding to it.

[0146] Similarly, in various instances, the illumination spot 1306 can be considered as being generated by or due to the reflective interaction of the reflector 318. In summary, the illumination spot 1306 can be considered as a visual representation of the beam 1204 captured by the camera 320, and the beam 1204 can be considered as originating from the beam 902 reflected by the reflector 318. Since the illumination spot 1306 can be generated by or due to the reflective interaction of the reflector 318, and since the reflector 318 can be considered as always being perpendicular or orthogonal to the axis-axis 406, the illumination spot 1306 can be considered as serving as a visual indication of the aperture-axis 406 or otherwise corresponding to it.

[0147] Therefore, since illumination spot 1304 can be considered a visual indicator of axis-axis 402, and since illumination spot 1306 (as an alternative) can be considered a visual indicator of aperture-axis 406, angular alignment or misalignment between axis-axis 402 and aperture-axis 406 can be visually or visibly represented in image 1302 as concentricity or non-concentricity between illumination spots 1304 and 1306. More specifically, when axis-axis 402 and aperture-axis 406 are angularly aligned (e.g., parallel to each other, but not necessarily collinear), such angular alignment will make illumination spots 1304 and 1306 concentric. In fact, when axis-axis 402 and aperture-axis 406 are parallel to each other, beam 902, after being reflected by reflector 318, can enter beam splitter 316 at the same location or point from which beam 902 originates. This causes beam 1102 to travel or propagate in the same direction as beam 702, which in turn causes beam 1204 to propagate in the same direction as beam 804. Therefore, beams 804 and 1204 can be considered as being projected onto the viewfinder of camera 320 at the same point or position as each other, which will make illumination spots 1304 and 1306 concentric in image 1302. In contrast, when axis-axis 402 and aperture-axis 406 are angularly misaligned (e.g., not parallel to each other), such angular misalignment will cause illumination spots 1304 and 1306 to be non-concentric. In short, when axis-axis 402 and aperture-axis 406 are not parallel to each other, beam 902, after being reflected by reflector 318, can enter beam splitter 316 at different positions or points from which beam 902 originates. This allows beam 1102 to travel or propagate in a different direction than beam 702, which in turn causes beam 1204 to propagate in a different direction than beam 804. Therefore, beams 804 and 1204 can be considered as being projected onto the viewfinder of camera 320 at different points or positions from each other, which will cause illumination spots 1304 and 1306 to be non-concentric in image 1302.

[0148] Figures 14 to 17 A non-limiting example of image 1302 is shown according to one or more embodiments described herein.

[0149] First, consider Figure 14 .like Figure 14 As shown in the non-limiting example, image 1302 may not depict any objects other than illumination spots 1304 and 1306. Figure 14 In the non-limiting example, illumination spot 1304 and illumination spot 1306 are not concentric with each other. In other words, Figure 14 Illumination spot 1304 and illumination spot 1306 are shown as having different positions within image 1302 (e.g., having different in-image positions from each other). Therefore, in Figure 14 In a non-limiting example, image 1302 can be interpreted as indicating that the axis-axis 402 and the hole-axis 406 are not angularly aligned (e.g., not parallel).

[0150] Now, consider Figure 15 .like Figure 15 As shown in the non-limiting example, image 1302 may still not depict any objects other than illumination spots 1304 and 1306. However, in Figure 15 In the non-limiting example, illumination spot 1304 and illumination spot 1306 are nearly concentric with each other. In other words, Figure 15 Illumination spot 1304 and illumination spot 1306 are shown as having almost identical positions within image 1302. Therefore, in Figure 15 In a non-limiting example, image 1302 can be considered as indicating that the axis-axis 402 and the hole-axis 406 are almost or mostly angularly aligned (e.g., almost or mostly parallel).

[0151] Now, consider Figure 16 .like Figure 16 As shown in the non-limiting example, illumination spots 1304 and 1306 overlap each other but are not yet concentric. Therefore, this can be seen as indicating the ratio of the axis-axis 402 to the hole-axis 406. Figure 14 The center is closer to the angle alignment, but compared to... Figure 15 The center is further off-center from the angle alignment.

[0152] In addition, such as Figure 16As shown in the non-limiting example, image 1302 may depict artifacts or residuals associated with illumination spot 1304 or illumination spot 1306, as indicated by reference numeral 1602. Such artifacts or residuals can be considered as unexpected, additional, or residual reflections or transmissions that may occur within beam splitter 316. As a non-limiting example, a portion of beam 604 may be reflected by the bottommost surface of beam splitter 316 (e.g., even though the bottommost surface has an applied anti-reflective coating), such that at least some of the reflected portion of beam 604 can pass through beam splitting coating 408 and thereby be guided toward camera 320. In this case, at least some of the reflected portion of beam 604 can visually appear as artifacts or residuals within image 1302.

[0153] In various situations, such as regarding Figure 17 As shown, such artifacts or residuals can be eliminated via thresholding. Specifically, given the rated power of light source 312 and the corresponding reflectivity and transmittance characteristics of beam splitter 316 and reflector 318, illumination spots 1304 and 1306 are expected to have at least an intensity threshold level when depicted or shown in image 1302. In contrast, any artifacts or residuals within beam splitter 316 that may be caused by unintended or residual reflections or transmissions are likely (alternatively) expected to be weaker than this intensity threshold level. Therefore, in various aspects, access component 328 can iterate over each pixel within image 1302. For any pixel with an intensity value greater than or equal to the intensity threshold level, the access component can keep that pixel unchanged. On the other hand, for any pixel with an intensity value less than the intensity threshold level, the access component can lower the intensity value of that pixel or reduce it to zero. In this way, access component 328 can be considered as removing or eliminating any artifacts or residuals in image 1302 without removing or eliminating illumination spot 1304 or illumination spot 1306. Therefore, image 1302 can be considered as not depicting any objects other than illumination spot 1304 and illumination spot 1306.

[0154] Note that in some cases, illumination spots 1304 and 1306 can be treated interchangeably without loss of generality. In other words, image 1302 can be considered as showing or depicting two illumination spots, and sometimes it is not necessary to know which of these two illumination spots corresponds to axis-axis 402 and (alternatively) which corresponds to aperture-axis 406. In short, at least in terms of angular alignment, the concentricity or non-concentricity between the two illumination spots is important. However, in other instances (e.g., regarding translational alignment such as that described later herein), illumination spots 1304 and 1306 can be considered non-interchangeable. In this case, the reflectivity and transmittance characteristics of beam splitter 316 and reflector 318 can be controlled or selected such that illumination spot 1304 is expected to have a different intensity than illumination spot 1306. Therefore, such intensity differences can be used to visually distinguish illumination spot 1304 from illumination spot 1306.

[0155] Figure 18 A block diagram of an exemplary non-limiting system including an object detection algorithm and overlap determination according to one or more embodiments described herein is shown, which facilitates the teaching of a charged particle microscope robotic gripper via beam splitting.

[0156] In various implementations, the overlay component 330 may electronically store, maintain, control, or otherwise access the object detection algorithm 1802. In various aspects, the object detection algorithm 1802 may be any suitable algorithm, model, or computerized program that counts or calculates the number of different or individual objects shown or depicted in an input image.

[0157] In some cases, the object detection algorithm 1802 can demonstrate the internal architecture of a deep learning neural network. In practice, the object detection algorithm 1802 can have an input layer, one or more hidden layers, and an output layer. In various instances, any of these layers can be coupled together by any suitable inter-neuron or inter-layer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, any of these layers can be any suitable type of neural network layer with any suitable learnable or trainable internal parameters. For example, any of these input layers, one or more hidden layers, or output layers can be convolutional layers, whose learnable or trainable parameters can be convolutional kernels. As another example, any of these input layers, one or more hidden layers, or output layers can be dense layers, whose learnable or trainable parameters can be weight matrices or bias values. As yet another example, any of these input layers, one or more hidden layers, or output layers can be batch normalization layers, whose learnable or trainable parameters can be translation factors or scaling factors. As yet another example, any of such an input layer, one or more hidden layers, or an output layer can be an LSTM layer, whose learnable or trainable parameters can be an input state weight matrix or a hidden state weight matrix. As yet another example, any of such an input layer, one or more hidden layers, or an output layer can be a transformer layer, whose learnable or trainable parameters can be single-head or multi-head attention blocks or other weight matrices. Furthermore, in various cases, any of such layers can be any suitable type of neural network layer with any suitable fixed or non-trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or an output layer can be a nonlinear layer, a padding layer, a pooling layer, or a cascaded layer.

[0158] In other cases, object detection algorithm 1802 can reveal any other suitable machine learning or artificial intelligence internal architecture. As a non-limiting example, in some aspects, object detection algorithm 1802 can reveal the internal architecture of a support vector machine. As another non-limiting example, in some aspects, object detection algorithm 1802 can reveal the internal architecture of a Naive Bayes. As yet another non-limiting example, in some aspects, object detection algorithm 1802 can reveal the internal architecture of a decision tree or random forest.

[0159] In other cases, object detection algorithm 1802 can demonstrate any other suitable non-machine learning internal architecture. As a non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that counts or statistically analyzes objects in an input image based on calculating or operating on the Hu moments of the input image. As another non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that counts or statistically analyzes objects in an input image based on calculating or operating on the pixel intensity curves of the input image. As yet another non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that counts or statistically analyzes objects in an input image based on calculating or operating on the oriented gradient histogram of the input image. As yet another non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that counts or statistically analyzes objects in an input image based on applying bounding box generation to the input image. As yet another non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that counts or statistically analyzes objects in an input image based on applying template matching to the input image. As another non-limiting example, object detection algorithm 1802 can be any suitable computer vision process or technique that is based on applying connected component analysis (CCA) to an input image to count or statistically analyze objects in the input image.

[0160] In some implementations, the object detection algorithm 1802 can be any suitable combination of any of the foregoing.

[0161] In any case, overlapping component 330 can utilize object detection algorithm 1802 to generate overlap determination 1804. Regarding Figure 19 The non-restrictive aspects are described.

[0162] Figure 19 An exemplary non-limiting block diagram is shown according to one or more embodiments described herein, illustrating how overlap determination 1804 is obtained.

[0163] In various implementations, the overlapping component 330 may electronically apply or execute object detection algorithm 1802 onto or on image 1302. In various aspects, such application or execution may produce object statistics 1902.

[0164] As a non-limiting example, imagine object detection algorithm 1802 demonstrating the internal architecture of a deep learning neural network. In this case, overlapping component 330 can feed image 1302 into the input layer of object detection algorithm 1802. In various instances, image 1302 can complete forward propagation through one or more hidden layers of object detection algorithm 1802. In various instances, the output layer of object detection algorithm 1802 can compute or operate object statistics 1902 based on activation maps or feature maps generated by one or more hidden layers.

[0165] As another non-limiting example, imagine object detection algorithm 1802 (as an alternative) exhibiting a non-machine learning architecture. In this case, overlapping component 330 can compute any Hu moments, pixel intensity curves, directional gradient histograms, or other image metrics or image features invoked by object detection algorithm 1802 for image 1302, and the end result of such computation can be object statistics 1902.

[0166] In any case, object statistics 1902 can be a scalar with a positive integer value or magnitude. In various aspects, the value or magnitude of object statistics 1902 can be considered as indicating, describing, or otherwise representing the number of distinct, discrete, or individual objects described or shown in image 1302 (as determined or inferred by object detection algorithm 1802). As mentioned above, image 1302 may (after applying thresholding processing, if applicable) not depict or show any objects other than illumination spots 1304 and 1306. Therefore, object statistics 1902 can be considered to have two possible values ​​or magnitudes: a value or magnitude 2, which may occur when illumination spots 1304 and 1306 are not visibly in contact with or overlap each other; and a value or magnitude 1, which may occur when illumination spots 1304 and 1306 (alternatively) are visibly in contact with or overlap each other.

[0167] In various respects, the overlap determination 1804 can be any suitable electronic data with any suitable format, size, or dimension (e.g., it can be one or more scalars, one or more vectors, one or more matrices, one or more tensors, a string, or any suitable combination thereof) that can indicate or describe whether illumination spot 1304 and illumination spot 1306 overlap. In various instances, the overlap component 330 can electronically generate the overlap determination 1804 based on reading object statistics 1902.

[0168] As a non-limiting example, imagine that object statistics 1902 have a value or magnitude of 2. This could mean that object detection algorithm 1802 has identified or counted two distinct, discrete, continuous, or separate objects within image 1302. Because image 1302 (due to alignment hardware 310) can be known to depict or show both illumination spots 1304 and 1306, without other objects, object statistics 1902 with a value or magnitude of 2 could indicate that illumination spots 1304 and 1306 appear visually separate or distinct within image 1302, meaning they do not overlap.

[0169] As another non-limiting example, imagine that object statistics 1902 (as an alternative) has a value or magnitude of 1. This could mean that object detection algorithm 1802 has identified or counted two distinct, separate, adjacent, or separate objects in image 1302. Since image 1302 (due to alignment hardware 310) can be known to depict or show both illumination spots 1304 and 1306, without other objects, object statistics 1902 with a value or magnitude of 1 could indicate that illumination spots 1304 and 1306 do not appear to be separate or distinct visually within image 1302, implying that they overlap.

[0170] Figure 20 A block diagram of an exemplary non-limiting system including a novel angular configuration according to one or more embodiments described herein is shown, which facilitates the teaching of a charged particle microscope robotic gripper via beam splitting.

[0171] In various implementations, the correction component 332 may electronically determine, electronically calculate, or otherwise electronically identify the new angular configuration 2002 based on the overlap determination 1804. In various aspects, the new angular configuration 2002 may be a specific value or state of the angular position control, parameters, or settings of the robot gripper 308, which is predicted or expected to cause the axis-axis 402 to become more angularly aligned with the hole-axis 406 (e.g., become more parallel). As a non-limiting example, consider... Figures 4 to 12 The X, Y, and Z axes are shown. In this case, the three-dimensional position and orientation of the robot gripper 308 within the vacuum chamber 304 can be quantified by the position of the robot gripper 308 (e.g., its end effector) along the X-axis, the position of the robot gripper 308 along the Y-axis, the position of the robot gripper 308 along the Z-axis, the rotation angle of the robot gripper 308 about the X-axis, the rotation angle of the robot gripper 308 about the Y-axis, and the rotation angle of the robot gripper 308 about the Z-axis. In various aspects, P XThis can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise affecting how the robot gripper 308 translates, moves, or otherwise displaces along the X-axis (e.g., how many units in the positive or negative direction). In various instances, P Y This can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise affecting how the robot gripper 308 translates, moves, or otherwise displaces along the Y-axis (e.g., how many units in the positive or negative direction). In various cases, P Z θ can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise affecting how the robot gripper 308 translates, moves, or otherwise displaces along the Z-axis (e.g., how many units in the positive or negative direction). In various aspects, θ X This can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise affecting how the robot gripper 308 rotates, turns, or otherwise moves about the X-axis (e.g., in the positive or negative direction, by how many angles or radians). In various instances, θ Y θ can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise influencing how the robot gripper 308 rotates, turns, or otherwise moves about the Y-axis (e.g., in the positive or negative direction, by how many angles or radians). In various cases, θ Z These can represent selectively controllable parameters of the robot gripper 308, indicating or otherwise influencing how the robot gripper 308 rotates, turns, or otherwise moves about the Z-axis (e.g., by how many angles or radians in the positive or negative direction). That is, the three-dimensional position and orientation of the robot gripper 308 at any given time can be controlled based on specific values, states, or configurations of the control vector using the following parameters: <P X P Y P Z θ X θ Y θ Z >

[0172] exist Figures 4 to 12 In a non-limiting example, the hole-axis 406 is fixed and parallel to the Y-axis. In this case, the angular alignment between the axis-axis 402 and the hole-axis 406 may depend on θ. X and θ Z These can be viewed as selectively controllable pitch settings / parameters and selectively controllable yaw settings / parameters for the robot gripper 308, respectively. In contrast, the angular alignment between axis-axis 402 and hole-axis 406 may not be affected by θ. Y P X P Y and P ZThe effects can be viewed as selectively controllable rolling settings / parameters, selectively controllable X-displacement settings / parameters, selectively controllable Y-displacement settings / parameters, and selectively controllable Z-displacement settings / parameters of the robot gripper 308, respectively. Therefore, in various instances, the new angle configuration 2002 can be θ. X and θ Z A specific value or state (e.g., a specific combination of controllable pitch and yaw values) will cause the axis-axis 402 to become angularly aligned with the hole-axis 406 (or at least become less angularly misaligned) if the robot gripper 308 exhibits or moves to that specific value or state.

[0173] In various aspects, how the correction component 332 determines, calculates, or identifies a new angular configuration 2002 may depend on the overlap determination 1804. Regarding Figures 21 to 26 Non-restrictive details are described.

[0174] Figures 21 to 26 An exemplary non-limiting block diagram is shown according to one or more embodiments described herein, illustrating how a new angular configuration 2002 can be identified based on overlap determination 1804.

[0175] First, consider Figure 21 . Figure 21 This relates to an overlap determination 1804 indicating or otherwise stating that illumination spots 1304 and 1306 do not overlap. In response to such an indication, correction component 332 may electronically calculate or compute the centroid 2102 of illumination spot 1304 and the centroid 2104 of illumination spot 1306 based on the respective pixel positions.

[0176] Specifically, each pixel of image 1302 can be considered to have a corresponding or unique intra-image position (e.g., with respect to row and column identifiers about a certain origin of image 1302). Furthermore, because illumination spots 1304 and 1306 do not overlap, pixels belonging to or constituting illumination spot 1304 of image 1302 can be visibly different, separate, isolated, or isolated from pixels belonging to or constituting illumination spot 1306 of image 1302. Therefore, in various aspects, correction component 332 can average the intra-image positions of all pixels belonging to or constituting illumination spot 1304, and the resulting average intra-image position can be considered as centroid 2102. Similarly, in various instances, correction component 332 can average the intra-image positions of all pixels belonging to or constituting illumination spot 1306, and the resulting average intra-image position can be considered as centroid 2104.

[0177] In various cases, the correction component 332 can then calculate the new angular configuration 2002 by applying any suitable pixel-to-physical kinematic formula or transformation to centroids 2102 and 2104 or to these centroids.

[0178] More specifically, the intra-image positions of centroids 2102 and 2104 can depend on the positions of beams 804 and 1204 projected onto the viewfinder of camera 320, respectively. Furthermore, the positions of beams 804 and 1204 projected onto the viewfinder of camera 320 can depend on the distance between beam splitter 316 and camera 320 (this distance can be considered known or knowable at any given moment because P...). Z It is selectively controllable, depending on the distance between the beam splitter 316 and the reflector 318 (this distance can be considered known or knowable at any given moment, because P...). Y (This is selectively controllable) and can depend on the individual incident and reflection angles used by the beam splitter 316 and reflector 318 to interact with beams 602, 702, 804, 902, 1102, and 1204. Furthermore, these different incident and reflection angles may initially be unknown, but they can be determined according to any suitable mathematical function (e.g., a combination of trigonometric functions) as θ... X and θ Z (For example, predictable geometric changes occur as the values ​​or states of the selectively controllable pitch and yaw settings / parameters assigned to the robot gripper 308 change.)

[0179] Now, because the distance between beam splitter 316 and camera 320 and reflector 318 at the current or present moment can be known (e.g., P...), Y and P Z The current or present value or state of θ, because θ X and θ Z The current or present value or state of the centroids 2102 and 2104 can be known, and since the current or present in-image positions of the centroids 2102 and 2104 can be calculated via pixel position averaging, the correction component 332 can use this known information and the aforementioned mathematical functions to calculate or estimate the current or present values ​​of the various incident and reflection angles of the beam splitter 316 and reflector 318 for interacting with beams 602, 702, 804, 902, 1102, and 1204. Given those calculated or estimated current or present values ​​of the incident and reflection angles, the aforementioned mathematical functions can be seen as informing the correction component 332 how the values ​​of the incident and reflection angles will or will change with θ. X and θ ZThe controlled changes (e.g., increasing the selectively controllable pitch setting / parameter of the robot gripper 308 by a first percentage or amount may be expected to decrease some incident and reflection angles by a second percentage or amount, and may be expected to increase other incident and reflection angles by a third percentage or amount). Similarly, since the positions of beams 804 and 1204 projected onto the viewfinder of camera 320 can depend on these different incident and reflection angles, the above mathematical function can be seen as informing the correction component 332 how the in-image positions of centroids 2102 and 2104 will, or will become, with respect to θ. X and θ Z The controlled changes occur. Therefore, such mathematical functions can be viewed as mapping the real-world physical space of the vacuum chamber 304 within which the robot gripper 308 can move to the two-dimensional pixel space of the image 1302, a term known as "pixel to physical".

[0180] Therefore, in various aspects, the correction component 332 can utilize those mathematical formulas (e.g., those pixel-to-physical kinematic transformations) to identify θ. X and θ Z The values ​​or states of θ, if occupied or implemented by the robot gripper 308, will cause centroids 2102 and 2104 to move toward or closer to each other within image 1302. In various instances, θ X and θ Z The values ​​or states identified in this way are referred to as the new angular configuration 2002.

[0181] Figure 22 A non-limiting example of image 1302 is shown, in which an executable... Figure 21 The calculation. In Figure 22 In a non-limiting example, illumination spots 1304 and 1306 do not overlap. Therefore, their pixels can be considered separate from each other or distinguishable. Thus, the intra-image positions of pixels belonging to illumination spot 1304 can be averaged, and this average intra-image position can be considered or taken as the position of centroid 2102. Similarly, the intra-image positions of pixels belonging to illumination spot 1306 can be averaged, and this average intra-image position can be considered or taken as the position of centroid 2104. Therefore, pixel-to-physical kinematics formulas can be applied to identify θ. X and θ Z Any value that will move the centroids 2102 and 2104 to a new position within the image where they will be closer to each other (e.g., to identify any value or state of the selectively controllable pitch and yaw settings / parameters of the robot gripper 308), and θ X and θ Z Such values ​​are referred to as the new angular configuration 2002.

[0182] Next, consider Figure 23 . Figure 23 This relates to an overlap determination 1804 indicating or otherwise indicating that illumination spots 1304 and 1306 overlap with each other (although the degree of overlap may be unknown). In response to such an indication, correction component 332 may electronically calculate or operate centroids 2102 and 2104 based on contour fitting.

[0183] Specifically, as described above, each pixel of image 1302 can be considered to have a corresponding or unique intra-image location (e.g., with respect to row and column identifiers about a certain origin of image 1302). However, because illumination spots 1304 and 1306 overlap, some pixels belonging to or constituting illumination spot 1304 in image 1302 may be visibly indistinguishable, not separated, not separated, or not isolated from pixels belonging to or constituting illumination spot 1306 in image 1302. In other words, due to the overlap, there will be some pixels belonging to both illumination spots 1304 and 1306, but identifying such pixels can be considered a significant or difficult computer vision task. Due to this complexity or difficulty, the centroids 2102 and 2104 cannot be reliably identified using pixel position averaging.

[0184] In various aspects, the correction component 332 can fit the image 1302 to the contour 2302 instead of using pixel position averaging. In various instances, the contour 2302 can be the boundary of any suitable geometry that defines the minimum area of ​​both illumination spots 1304 and 1306 (e.g., a rectangular boundary, an elliptical boundary, any other polygonal boundary). Here, "minimum area" can mean that the contour 2302 defines as few pixels of image 1302 as possible, while still taking into account its geometry, and that it still defines all pixels belonging to at least one of illumination spots 1304 and 1306.

[0185] It should be noted that the area of ​​contour 2302 may depend on how far apart illumination spot 1304 and illumination spot 1306 are. For example, for any geometry of contour 2302, its area may decrease as illumination spot 1304 and illumination spot 1306 get closer to each other, and its area may (instead) increase as illumination spot 1304 and illumination spot 1306 get further apart from each other.

[0186] In various cases, the correction component 332 can generate or calculate the contour 2302 via any suitable computer vision contour fitting technique.

[0187] In some respects, the correction component 332 can generate the contour 2302 via any suitable machine learning or artificial intelligence technique. As a non-limiting example, the correction component 332 can electronically store, maintain, control, or otherwise access the deep learning neural network. In various cases, the deep learning neural network can have an input layer, one or more hidden layers, and an output layer. In various instances, any of such layers can be coupled together by any suitable inter-neuron or inter-layer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, any of such layers can be any suitable type of neural network layer with any suitable learnable or trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or output layer can be a convolutional layer, whose learnable or trainable parameters can be convolutional kernels. As another example, any of such an input layer, one or more hidden layers, or output layer can be a dense layer, whose learnable or trainable parameters can be weight matrices or bias values. As yet another example, any of such an input layer, one or more hidden layers, or output layer can be a batch normalization layer, whose learnable or trainable parameters can be translation factors or scaling factors. As yet another example, any of such an input layer, one or more hidden layers, or an output layer can be an LSTM layer, whose learnable or trainable parameters can be an input state weight matrix or a hidden state weight matrix. As yet another example, any of such an input layer, one or more hidden layers, or an output layer can be a transformer layer, whose learnable or trainable parameters can be single-head or multi-head attention blocks or other weight matrices. Furthermore, in various cases, any of such layers can be any suitable type of neural network layer with any suitable fixed or non-trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or an output layer can be a nonlinear layer, a padding layer, a pooling layer, or a cascaded layer.

[0188] Regardless of its specific internal architecture, a deep learning neural network can be configured to define the illumination spot depicted in the input image using a minimum area contour shape. Therefore, in various aspects, the correction component 332 can perform a deep learning neural network operation on the image 1302, and such operation can cause the deep learning neural network to generate a contour 2302. For example, the correction component 332 can feed the image 1302 into the input layer of the deep learning neural network, the image 1302 can undergo forward propagation through one or more hidden layers of the deep learning neural network, and the output layer of the deep learning neural network can compute or operate on the contour 2302 based on the activation map or feature map generated by the one or more hidden layers.

[0189] In other respects, the correction component 332 may (alternatively) generate the contour 2302 via any suitable non-machine learning computer vision technique. As a non-limiting example, the correction component 332 may generate the contour 2302 by applying any suitable computer vision process or technique utilizing convex hull computation (e.g., Graham's Scan algorithm) to the image 1302. As another non-limiting example, the correction component 332 may generate the contour 2302 by applying any suitable computer vision process or technique utilizing polygon approximation (e.g., Douglas-Peucker algorithm) to the image 1302. As yet another non-limiting example, the correction component 332 may generate the contour 2302 by applying any suitable computer vision process or technique utilizing least-squares fitting computation (e.g., least-squares ellipse fitting, least-squares rectangle fitting) to the image 1302.

[0190] In some implementations, the correction component 332 may generate profile 2302 via any suitable combination of the foregoing.

[0191] In any case, contour 2302 can be considered to have a known in-image location and a known fitted size (e.g., known length, known width). In various aspects, correction component 332 can identify centroids 2102 and 2104 by applying any suitable geometric formula or equation to contour 2302. In fact, given the geometry of contour 2302, the radii of illumination spots 1304 and 1306 can have any suitable known geometric relationship with contour 2302. Therefore, correction component 332 can utilize the fitted size of contour 2302 and this known geometric relationship to identify the in-image locations of centroids 2102 and 2104. As a non-limiting example, imagine contour 2302 is rectangular, imagine illumination spots 1304 and 1306 have the same radius, and imagine illumination spots 1304 and 1306 overlap in the length direction of contour 2302. In this case, contour 2302 can be considered to have a fitted length and a fitted width, and it can be expected that the fitted width of contour 2302 is twice its radius. Therefore, the correction component 332 can: identify the midpoint of one of the fitting width sides of the contour 2302; starting from this midpoint, traverse half the fitting width in a direction toward the interior of the contour 2302 and parallel to the fitting length side of the contour 2302; and any point reached by such traversal can be considered as one of the centroids 2102 and 2104. Conversely, the correction component 332 can: identify the midpoint of the other fitting width side of the contour 2302; starting from this midpoint, traverse half the fitting width in a direction toward the interior of the contour 2302 and parallel to the fitting length side of the contour 2302; and any point reached by such traversal can be considered as the other of the centroids 2102 and 2104.

[0192] The inventors now recognize that when centroids 2102 and 2104 are far apart (e.g., when illumination spots 1304 and 1306 do not overlap or only slightly overlap), calculating the new angular configuration 2002 using pixel-to-physical kinematics formulas may be more efficient or reliable. In contrast, the inventors recognize that when centroids 2102 and 2104 are close together (e.g., when illumination spots 1304 and 1306 extensively overlap), calculating the new angular configuration 2002 using pixel-to-physical kinematics formulas may (as an alternative) be less efficient or reliable. Therefore, in various aspects, the correction component 332 can calculate the separation distance 2304 between centroids 2102 and 2104. As a non-limiting example, the separation distance 2304 may be equal to or otherwise based on the Euclidean distance between centroids 2102 and 2104. As another non-limiting example, the separation distance 2304 may be equal to or otherwise based on the number of intermediate pixels between centroids 2102 and 2104. In various cases, the correction component 332 may recognize the new angular configuration 2002 based on the separation distance 2304.

[0193] consider Figure 24 In various implementations, as shown in the figure, the way the correction component 332 identifies the new angular configuration 2002 may depend on the separation distance 2304. In particular, the correction component 332 may compare the separation distance 2304 with any suitable threshold distance, and the correction component 332 may calculate the new angular configuration 2002 based on such comparison.

[0194] In all respects, it is assumed that the separation distance 2304 is greater than a threshold distance. In response, the correction component 332 can conclude that the centroids 2102 and 2104 are far enough apart that the pixel-to-physical kinematics formula can be used effectively or reliably. Therefore, as shown in the figure, as described above, the correction component 332 can calculate or identify a new angular configuration 2002 by applying the pixel-to-physical kinematics formula to the centroids 2102 and 2104.

[0195] In contrast, it is assumed that the separation distance 2304 is less than this threshold distance. In response, the correction component 332 can conclude that the centroids 2102 and 2104 are close enough that the pixel-to-physical kinematics formula cannot be used effectively or reliably. In this case, the correction component 332 can electronically command, electronically instruct, or otherwise electronically move the robot gripper 308 through or sweep multiple sweep angle configurations 2402. In various instances, the multiple sweep angle configurations 2402 may include n configurations, where n is any suitable positive integer: sweep angle configuration 2402(1) is updated to sweep angle configuration 2402(n). In various cases, each of the multiple sweep angle configurations 2402 can be θ. X and θ Z The unique combination of values ​​or states in θ X and θ Z The sweep angle configuration 2402(1) can be the first unique combination of values ​​or states of the selectively controllable pitch and yaw settings / parameters of the robot gripper 308 that are within a threshold proximity or margin of the values ​​or states currently or at any given moment. As a non-limiting example, the sweep angle configuration 2402(n) can be the nth unique combination of values ​​or states of the selectively controllable pitch and yaw settings / parameters of the robot gripper 308 that are within a threshold proximity or margin of the values ​​or states currently or at any given moment. In some cases, the threshold proximity or margin can be considered as an angle or orientation range or can be considered as defining an angle or orientation range with respect to θ. X and θ Z Centered on the current or present value or state, the multiple sweep angle configurations 2402 can be evenly spaced or evenly distributed across this range. However, in other cases, the multiple sweep angle configurations 2402 can be non-uniformly spaced or non-uniformly distributed.

[0196] In any case, the correction component 332 can cause the robot gripper 308 to move temporarily or momentarily or otherwise present each of the plurality of sweep angle configurations 2402. It should be noted that such movement can change the incident and reflection angles of the beams 602, 702, 804, 902, 1102, and 1204 imparted by the beam splitter 316 and reflector 318. Therefore, such movement can change the in-image positions of the centroids 2102 and 2104, as well as the in-image position and fit size of the contour 2302. In various aspects, the correction component 332 can calculate or compute a corresponding area of ​​the contour 2302 (e.g., the corresponding area defined by the contour) for each of the plurality of sweep angle configurations 2402, thereby producing a plurality of resulting contour areas 2404. In practice, the plurality of resulting contour areas 2404 can each correspond to a plurality of sweep angle configurations 2402. That is, since the multiple sweep angle configurations 2402 may include n configurations, the multiple resulting profile areas 2404 may include n areas: resulting profile area 2404(1) to resulting profile area 2404(n). As a non-limiting example, the resulting profile area 2404(1) may be a measured or approximate area defined by profile 2302, which is caused by any fitted size of profile 2302 when the robot gripper 308 moves to or presents the sweep angle configuration 2402(1). As another non-limiting example, the resulting profile area 2404(n) may be a measured or approximate area defined by profile 2302, which is caused by any fitted size of profile 2302 when the robot gripper 308 moves to or presents the sweep angle configuration 2402(n).

[0197] In various aspects, the correction component 332 can identify a new angular configuration 2002 based on a plurality of resulting profile areas 2404. As a non-limiting example, the new angular configuration 2002 can be any of a plurality of swept angular configurations 2402 that has or corresponds to the lowest, minimum, or otherwise minimized resulting profile area. In summary, if a particular angular configuration (e.g., θ) X and θ Z A specific value) makes the profile 2302 have a smaller area than any other angular configuration, which can be regarded as achieving maximum overlap between the illumination spot 1304 and the illumination spot 1306, and thereby achieving maximum or optimal angular alignment between the axis-axis 402 and the hole-axis 406.

[0198] Figures 25 to 26 A non-limiting example of image 1302 is shown, in which implementation is possible. Figures 23 to 24 The calculation.

[0199] First, consider Figure 25 .exist Figure 25In a non-limiting example, illumination spots 1304 and 1306 overlap, but such overlap is not widespread. Because of this overlap, their pixels cannot be considered separate or distinguishable from each other. Therefore, implementing pixel position averaging may be less reliable or straightforward. Instead, image 1302 can be fitted with contour 2302 to define both illumination spots 1304 and 1306 while still having a minimum area, and centroids 2102 and 2104 can be geometrically estimated based on the fitted dimensions of contour 2302. Figure 25 In the non-restricted example, centroids 2102 and 2104 can be considered as being separated by a distance greater than the threshold. Therefore, pixel-to-physical kinematics formulas can be applied to identify θ. X and θ Z Any value that will move the centroids 2102 and 2104 to a new position within the image where they will be closer to each other (e.g., to identify any value or state of the selectively controllable pitch and yaw settings / parameters of the robot gripper 308), and θ X and θ Z Such values ​​are referred to as the new angular configuration 2002.

[0200] Next, consider Figure 26 .exist Figure 26 In a non-limiting example, illumination spots 1304 and 1306 overlap, and this overlap is extensive. Because of this overlap, their pixels cannot be considered separate or distinguishable from each other. Therefore, implementing pixel position averaging may be less reliable or straightforward. Instead, image 1302 can be fitted with a contour 2302 to define both illumination spots 1304 and 1306 while still having a minimum area, and centroids 2102 and 2104 can be geometrically estimated based on the fitted dimensions of contour 2302. Figure 26 In a non-limiting example, centroids 2102 and 2104 can be considered as being separated by a distance less than a threshold. Therefore, pixel-to-physical kinematics formulas can be bypassed. Instead, the robot gripper 308 can sweep any suitable range of angular configurations (e.g., θ). X and θ Z The value), that is, centered on its current angular configuration (e.g., its current θ). X and θ Z The value is centered), and the corresponding area of ​​profile 2302 can be calculated for each of these sweep angle configurations. In various cases, any of these sweep angle configurations, as long as it minimizes the area of ​​profile 2302, can be considered a new angle configuration 2002.

[0201] In various embodiments, the execution component 334 may electronically execute or initiate any suitable electronic action based on the new angular configuration 2002. As a non-limiting example, the execution component 334 may electronically instruct, command, or otherwise electronically move the robot gripper 308 to or present the new angular configuration 2002. As another non-limiting example, the execution component 334 may (e.g., in any suitable database or data structure) electronically mark or identify the new angular configuration 2002 as angularly aligned with the aperture 404 of the microscope mesh container 306. For example, in some cases, the execution component 334 may electronically notify the charged particle microscope 302 that the new angular configuration 2002 has been determined or verified to be angularly aligned with the aperture 404 of the microscope mesh container 306. Thus, the charged particle microscope 302 may be considered to now know how to angularly align the robot gripper 308 with the aperture 404 of the microscope mesh container 306 for future reference.

[0202] Figures 27 to 30 Flowcharts are shown of exemplary, non-limiting computer-implemented methods 2700, 2800, 2900, and 3000 according to one or more embodiments described herein, which facilitate teaching a charged particle microscope robotic gripper via beam splitting. In various cases, system 322 may facilitate or perform computer-implemented methods 2700, 2800, 2900, or 3000.

[0203] First, consider Figure 27 In various embodiments, action 2702 may include: accessing a charged particle microscope (e.g., 302) having a vacuum chamber (e.g., 304) via a device (e.g., via 328) operably coupled to a processor (e.g., 324), wherein a robotic gripper (e.g., 308) and a mesh container (e.g., 306) are located inside the vacuum chamber.

[0204] In various aspects, action 2704 may include accessing an image (e.g., 1302) captured by a camera (e.g., 320) of a charged particle microscope via a device (e.g., via 328), wherein the image may depict a first spot (e.g., 1304) and a second spot (e.g., 1306), wherein a light source (e.g., 312) coupled to a robotic gripper may emit a beam (e.g., 502) through a beam splitter (e.g., 316), wherein the beam splitter may direct a first portion (e.g., 804) of the beam toward the camera to form a first spot, wherein the beam splitter may direct a second portion (e.g., 902) of the beam toward an aperture (e.g., 404) in a mesh container, wherein a reflector (e.g., 318) in the aperture may direct the second portion of the beam back to the beam splitter, and wherein the beam splitter may direct at least some of the second portion of the beam (e.g., 1204) toward the camera to form a second spot.

[0205] In various instances, action 2706 may include applying object detection techniques (e.g., 1802) to an image via a device (e.g., via 330). In various cases, computer-implemented method 2700 may proceed to action 2802 of computer-implemented method 2800.

[0206] Now, consider Figure 28 In various embodiments, action 2802 may include determining, by means of a device (e.g., via 332), whether the first spot and the second spot overlap. In other words, the device may determine, via object detection technology, whether the image depicts two separate objects or only one object. If the first spot and the second spot overlap (e.g., if the object detection technology identifies only one object in the image), then the computer-implemented method 2800 may proceed to action 2902 of the computer-implemented method 2900. Conversely, if the first spot and the second spot do not overlap (e.g., if the object detection technology identifies two different or separate objects in the image), then the computer-implemented method 2800 may proceed to action 2804.

[0207] In various aspects, action 2804 may include identifying the first centroid of the first spot by averaging the positions of any pixels of the image belonging to the first spot using a device (e.g., via 332) (e.g., 2102).

[0208] In various instances, action 2806 may include identifying the second centroid of the second spot by averaging the positions of any pixels in the image that belong to the second spot using a device (e.g., via 332) (e.g., 2104).

[0209] In various cases, action 2808 may include identifying, via a device (e.g., via 332) and via a pixel-to-physical kinematics formula corresponding to a charged particle microscope, an angular configuration of the robot gripper that will move the first and second centers of mass closer to each other (e.g., 2002).

[0210] In various aspects, action 2810 may include moving the robot gripper to the identified angular configuration via a device (e.g., via 334).

[0211] Next, consider Figure 29 In various embodiments, action 2902 may include fitting an image to a contour (e.g., 2302) via a device (e.g., via 332) such that the contour is the smallest area shape defining both the first spot and the second spot.

[0212] In various aspects, action 2904 may include identifying the first centroid of the first spot by means of a device (e.g., via 332) using a geometric formula for a profile-based fitted size.

[0213] In various instances, action 2906 may include identifying the second centroid of the second spot by means of a device (e.g., via 332) using a geometric formula for a profile-based fitted size.

[0214] In various cases, action 2908 may include calculating the distance between the first centroid and the second centroid via a device (e.g., via 332) (e.g., 2304).

[0215] In various aspects, action 2910 may include determining, via a device (e.g., via 332), whether the distance is greater than a threshold. If the distance is not greater than the threshold, the computer-implemented method 2900 may proceed to action 3002 of the computer-implemented method 3000. Conversely, if the distance is greater than the threshold, the computer-implemented method 2900 may proceed to action 2912.

[0216] In various instances, action 2912 may include action 2808 of computer-implemented method 2800 performed via a device (e.g., via 332).

[0217] Now, consider Figure 30 In various embodiments, action 3002 may include sweeping the robot gripper within a threshold proximity of the robot gripper’s current angular configuration (e.g., 2402) by means of a device (e.g., via 332).

[0218] In various aspects, action 3004 may include calculating the area of ​​the profile (e.g., a corresponding one in 2404) for each of a plurality of angular configurations via a device (e.g., via 332).

[0219] In various instances, action 3006 may include moving the robot gripper via a device (e.g., via 334) to any of a plurality of angular configurations corresponding to the minimum or minimized area of ​​the profile (e.g., 2002).

[0220] As described herein, various implementations can be achieved via machine learning or artificial intelligence (e.g., object detection algorithm 1802 can be a deep learning neural network; correction component 332 can utilize a deep learning neural network to generate contour 2302). To ensure reliable operation of such implementations, such machine learning or artificial intelligence can first be trained. Regarding Figure 31 An unrestricted example of this type of training is described.

[0221] Figure 31 An exemplary non-limiting block diagram is shown according to one or more embodiments described herein, illustrating how a machine learning model can be trained.

[0222] In various aspects, a machine learning model 3102 may exist. In various instances, the machine learning model 3102 may exhibit any suitable internal architecture as described above (e.g., input layer, hidden layer, and output layer, any of which may be a convolutional layer, LSTM layer, dense layer, or nonlinear layer, and any of which may be coupled by any suitable inter-layer connections, such as forward connections, recurrent connections, or skip connections). In some cases, the machine learning model 3102 may be considered as a neural network facilitating the object detection algorithm 1802. In other cases, the machine learning model 3102 may be considered as a neural network capable of computing contours 2302. In any case, the trainable internal parameters of the machine learning model 3102 (e.g., convolutional kernels, weight matrices, bias values) may be initialized in any suitable manner (e.g., via random initialization) before training begins.

[0223] In various implementations, a training image 3104 and a benchmark ground truth annotation 3106 may exist. If the machine learning model 3102 is designed, intended, or configured to facilitate the object detection algorithm 1802, the training image 3104 may be any suitable image with the same format, size, or dimensions as image 1302 and depict any two illumination spots that may or may not overlap each other, and the benchmark ground truth annotation 3106 may be any correct or accurate object statistics (e.g., the correct or accurate number of distinct, individual objects) known or considered to be depicted or shown in the training image 3104. If the machine learning model 3102 (as an alternative) is designed, intended, or configured to facilitate contour fitting, the training image 3104 may be any suitable image with the same format, size, or dimensions as image 1302 and depict any two illumination spots that overlap each other, and the benchmark ground truth annotation 3106 may be any correct or accurate contour that has the same shape as contour 2302, is known or considered to have a minimum area, and defines the two illumination spots depicted in the training image 3104.

[0224] In any case, the machine learning model 3102 can be executed on the training image 3104, thereby causing the machine learning model 3102 to produce an output 3108. For example, in some cases, the training image 3104 can be fed or routed to the input layer of the machine learning model 3102, the training image 3104 can be forward-propagated through one or more hidden layers of the machine learning model 3102, and the output layer of the machine learning model 3102 can compute the output 3108 based on the activation maps or feature maps provided by one or more hidden layers of the machine learning model 3102.

[0225] It should be noted that the format, size, or dimension of output 3108 can be determined by the number, arrangement, size, or other features of neurons, convolutional kernels, LSTM layers, or other internal parameters of the output layer (or any other layer) of machine learning model 3102. Therefore, by adding, deleting, or otherwise adjusting features of the output layer (or any other layer) of machine learning model 3102, output 3108 can be forced to have any desired format, size, or dimension.

[0226] In all respects, if the machine learning model 3102 is designed, configured, or intended to facilitate the object detection algorithm 1802, then the output 3108 can be considered as the predicted or inferred object statistics that the machine learning model 3102 believes should correspond to the training image 3104. If the machine learning model 3102 (as an alternative) is designed, configured, or intended to facilitate contour fitting, then the output 3108 can be considered as the minimum area spot-defined contour that the machine learning model 3102 believes should correspond to the training image 3104. It should be noted that if the machine learning model 3102 has not been trained or has been trained very little to date, then the output 3108 may be highly inaccurate. In other words, the output 3108 may differ significantly from the baseline true annotation 3106.

[0227] In all respects, the error 3110 (e.g., mean absolute error, mean squared error, cross-entropy error) between the output 3108 and the baseline true annotation 3106 can be computed. In various instances, the trainable intrinsic parameters of the machine learning model 3102 can be progressively updated based on the error 3110 via backpropagation (e.g., stochastic gradient descent).

[0228] In various cases, this execution and update procedure can be repeated for any suitable number of training images. This ultimately allows for iterative optimization of the trainable intrinsic parameters of the machine learning model 3102 to accurately perform its inference task (e.g., object statistics or contour fitting). In all respects, any suitable training batch size, any suitable error / loss function, or any suitable training termination criterion can be utilized during such training.

[0229] Although the disclosure herein primarily describes the supervised training of machine learning model 3102, this is merely a non-limiting example for ease of explanation and illustration. In various implementations, any other suitable training paradigm can be used to train machine learning model 3102, such as unsupervised training or reinforcement learning, either jointly or non-jointly.

[0230] Figures 32 to 36 Exemplary non-limiting images of various experimental practices according to one or more embodiments described herein are shown.

[0231] First, consider Figure 32 . Figure 32A computer-aided design drawing 3200 of the grid support 3202 is shown. The grid support 3202 can be considered as a structural accessory that can physically hold, carry, or support two microscope grids: a first grid at the location indicated by reference numeral 3204; and a second grid at the location indicated by reference numeral 3206. As shown, there is an axis 3208 that protrudes from the grid support 3202. In various aspects, a robotic gripper 308 can grip or hold the grid support 3202. When the robotic gripper 308 grips or holds the grid support 3202, the axis 3208 can be considered to extend longitudinally along the axis-axis 402. It should be noted that when equipped with two microscope grids, the total mass of the grid support 3202 can be approximately 1.3 grams, as measured by the inventors.

[0232] Now, consider Figure 33 . Figure 33 Computer-aided design drawing 3300 illustrates a non-limiting exemplary embodiment of a microscope mesh container 306. As shown, the microscope mesh container 306 may have a hole 404. A shaft 3208 may be sized to be slidably fitted inside the hole 404. When the shaft 3208 is slidably inserted into the hole 404, the mesh support 3202 can be considered as temporarily attached to the microscope mesh container 306. Because there may be a very small gap between the shaft 3208 and the hole 404, it may be desirable to teach the robotic gripper 308 how to angularly align the shaft 3208 (e.g., shaft-axis 402) with the hole 404 (e.g., hole-axis 406).

[0233] Next, consider Figure 34 . Figure 34 Photograph 3400 shows a prototype constructed by the inventors according to various embodiments described herein. As shown, the prototype includes a light source 312 (e.g., a printed circuit board having the light source 312 fabricated thereon), a collimating lens 314 (e.g., a metal block or housing containing the collimating lens 314), and a beam splitter 316, all coupled together via a mounting bracket 3402. Also shown, the light source 312 may be powered by wires 3404.

[0234] Figure 35 An enlarged view of the light source 312, collimating lens 314, and beam splitter 316 mounted on the mounting bracket 3402 is shown. Figure 35 The specific configuration shown has a total mass of 2 grams, as measured by the inventors. It should be noted that such a 2-gram mass is considered equivalent to (e.g., on the same order of magnitude) the 1.3-gram mass of the mesh support 3202. Therefore, when held or clamped... Figure 35In the configuration shown, the robotic gripper 308 can be considered to exhibit no significantly greater deflection compared to when the mesh support 3202 is held or clamped. In stark contrast, some existing technologies require holding or clamping most available miniature cameras or pipe mirrors weighing tens of grams, which could result in significantly greater deflection for the robotic gripper 308 compared to holding or clamping the mesh support 3202.

[0235] Finally, consider Figure 36 . Figure 36 (via reference numeral 3602) shows a light source 312, a collimating lens 314, and a beam splitter 316 (all mounted on the now-obscured mounting bracket 3402) held or held by a non-limiting exemplary embodiment of a robotic gripper 308. Figure 36 A non-limiting exemplary embodiment is also shown, in which the reflector 318 is vertically attached to the outside of the aperture 404 of the microscope grid container 306. (See also:) Figure 36 As shown, the light source 312, collimating lens 314, and beam splitter 316 can be considered as pointing to the reflector 318. Therefore, as regarding Figures 4 to 12 The light beam generated by the light source 312 can be transmitted or reflected by the beam splitter 316 and the reflector 318.

[0236] Figure 36 This can be considered a real-world experimental setup depicting the various embodiments described herein. The inventors have demonstrated that this real-world experimental setup achieves angular alignment between the robot gripper 308 and the hole 404 with a resolution of less than 1 arcsecond per pixel, which can be considered highly precise angular alignment. This type of experimental setup verifies the specific, real-world technical benefits achieved by the various embodiments described herein.

[0237] While the disclosure herein has primarily described various embodiments that allow the shaft-axis 402 to be angularly aligned (e.g., parallel) with the hole-axis 406, these are merely non-limiting examples. In some instances, the various embodiments described herein may also be implemented to make the shaft-axis 402 translatively aligned (e.g., collinear) with the hole-axis 406. In various respects, as described above, the angular alignment between the shaft-axis 402 and the hole-axis 406 may depend on θ X and θ Z (For example, this may depend on the values ​​or states of the selectively controllable pitch and yaw settings / parameters of the robot gripper 308). In contrast, the translational alignment between axis-axis 402 and hole-axis 406 may depend on P X and P Z (For example, it may depend on the value or state of the selective controllable X and Z displacement settings / parameters of the robot gripper 308).

[0238] In various cases, reflector 318 may have a spatially varying reflectivity. In other words, the amount or intensity of incident light reflected by reflector 318 may be spatially non-uniform, depending on the location of such incident light projected onto the reflector. In various cases, such non-uniform reflection can be visually represented as the intensity of illumination spot 1306. In other words, given a variation in the non-uniform reflectivity of reflector 318, and given the current or present intensity of illumination spot 1306, correction component 332 can determine, identify, or otherwise infer where beam 902 (and thus axis-axis 402) is projected onto the surface of reflector 318. Therefore, correction component 332 can accordingly cause robot gripper 308 to sweep any suitable range of translational configurations (e.g., sweep any suitable range of P). X and P Z The value or state), this range is centered on the translation configuration at the current or present moment (e.g., with P). X and P Z The current or present value is centered. For each of the sweep translation configurations, the correction component 332 can measure the intensity (e.g., maximum or average intensity) of the illumination spot 1306. In various instances, the center of the surface of the reflector 318 may have a unique or distinctive reflectivity, or be associated with it. Therefore, any of the sweep translation configurations that causes the intensity of the illumination spot 1306 to reach or approach any intensity value expected to be caused by the unique or distinctive reflectivity of the center of the reflector 318 can be considered as achieving translational alignment between the axis-axis 402 and the aperture-axis 406.

[0239] As a non-limiting example, reflector 318 may include multiple annular reflective rings, each ring having a lower reflectivity compared to its radially inner neighboring ring, and each ring having a higher reflectivity compared to its radially outer neighboring ring. Therefore, the reflectivity of reflector 318 can be considered to be at a maximum value at the center of its surface, gradually decreasing radially outward from that center. In this case, correction component 332 can determine that translational alignment has been achieved as long as the intensity of illumination spot 1306 is higher than any threshold associated with the maximum reflectivity at the center of reflector 318. Conversely, correction component 332 can determine that translational alignment has not been achieved as long as the intensity of illumination spot 1306 is less than any threshold associated with the maximum reflectivity at the center of reflector 318.

[0240] In some cases, the intensity-based techniques described above for image 1302 and the spatially varying reflectivity of reflector 318 can be combined with touch alarm techniques to achieve more precise translational alignment between axis-axis 402 and aperture-axis 406. In various aspects, touch alarm techniques can be implemented after performing the intensity-based techniques described above. More specifically, touch alarm techniques may involve: coupling a touch alarm axis to a robot gripper 308 that extends longitudinally along axis-axis 402 and has a radius smaller than the radius of the axis protruding from the microscope grid; and applying electrical bias voltages to the robot gripper 308 and the microscope grid container 306 at different voltages (e.g., the robot gripper 308 may be electrically grounded, and the microscope grid container 306 may be biased at 0.5 volts). In this configuration, the robot gripper 308 can move to any translational configuration that, according to the aforementioned strength-based technique, achieves translational alignment between axis-axis 402 and hole-axis 406. The robot gripper 308 can incrementally translate upwards from this configuration until electrical continuity between the robot gripper 308 and the microscope mesh container 306 is detected (e.g., until the touch alarm axis physically contacts the inner surface of the hole 404). When this electrical continuity is detected, the spatial coordinates of the robot gripper 308 can be considered as marking the upper limit of translational alignment. Similarly, the robot gripper 308 can move to any translational configuration that, according to the aforementioned strength-based technique, achieves translational alignment between axis-axis 402 and hole-axis 406. The robot gripper 308 can incrementally translate downwards from this configuration until electrical continuity between the robot gripper 308 and the microscope mesh container 306 is detected. When this electrical continuity is detected, the spatial coordinates of the robot gripper 308 can be considered as marking the lower limit of translational alignment. Similarly, the robot gripper 308 can be moved to any translational configuration, as described above, based on the strength-based technique, that achieves translational alignment between the axis-axis 402 and the hole-axis 406. The robot gripper 308 can be translated incrementally to the right from this translational configuration until an electrical continuity between the robot gripper 308 and the microscope mesh container 306 is detected. When such an electrical continuity is detected, the spatial coordinates of the robot gripper 308 can be considered to mark the right limit of the translational alignment. Likewise, the robot gripper 308 can be moved to any translational configuration, as described above, based on the strength-based technique, that achieves translational alignment between the axis-axis 402 and the hole-axis 406. The robot gripper 308 can be translated incrementally to the left from this translational configuration until an electrical continuity between the robot gripper 308 and the microscope mesh container 306 is detected. When such an electrical continuity is detected, the spatial coordinates of the robot gripper 308 can be considered to mark the left limit of the translational alignment. Therefore, the center or midpoint of the upper limit of translation alignment, the lower limit of translation alignment, the right limit of translation alignment, and the left limit of translation alignment can be regarded as the precise center of hole 404.Therefore, precise translational alignment can be achieved by having the robot gripper 306 place the axis-axis 402 at the center or midpoint.

[0241] Although the disclosure herein primarily describes various embodiments, in which angular misalignment between axis-axis 402 and aperture-axis 406 is indicated by the non-concentricity of illumination spot 1304 and illumination spot 1306, and in which angular alignment between axis-axis 402 and aperture-axis 406 is indicated by the concentricity of illumination spot 1304 and illumination spot 1306, these are merely non-limiting examples for ease of interpretation. It should be understood and recognized that while some applications or uses of the charged particle microscope 302 may require or necessitate simultaneous alignment of axis-axis 402 and aperture-axis 406 in both the pitch and yaw directions, other applications or uses of the charged particle microscope 302 may require or necessitate alignment of axis-axis 402 and aperture-axis 406 only in one of the pitch and yaw directions (e.g., aligned in the pitch direction but not in the yaw direction; or aligned in the yaw direction but not in the pitch direction). These applications or uses can be considered as requiring only partial angular alignment between axis-axis 402 and aperture-axis 406. In various cases, system 322 can achieve this partial angular alignment by recognizing (e.g., via any suitable pixel to a physical kinematic formula) the angular configuration of the robot gripper 308 that enables illumination spots 1304 and 1306 to achieve some specific but non-concentric layout or arrangement.

[0242] As a non-limiting example, without loss of generality, it is conceivable that adjusting the pitch of the robotic gripper 304 changes the intra-image pixel row coordinates of centroids 2102 and 2104; and adjusting the yaw of the robotic gripper 304 changes the intra-image pixel column coordinates of centroids 2102 and 2104. In this case, adjusting the angular configuration of the robotic gripper 308 to make illumination spots 1304 and 1306 concentric can be considered equivalent to making centroids 2102 and 2104 have the same intra-image pixel row and column coordinates. Now, some applications or uses of the charged particle microscope 302 may only require pitch angle alignment. Such pitch angle alignment can be achieved by identifying (e.g., via any suitable pixel-to-physical kinematic formula) the angular configuration of the robotic gripper 308 that makes centroids 2102 and 2104 have the same intra-image pixel row coordinates (regardless of their intra-image pixel column coordinates). Therefore, in this example, partial angular alignment (e.g., alignment in the pitch direction but not in the yaw direction) can be achieved, but not by making illumination spots 1304 and 1306 concentric, but rather by making their centroids have the same in-image pixel row coordinates. Conversely, other applications or uses of the charged particle microscope 302 may only require yaw angle alignment. Such yaw angle alignment can be achieved by recognizing (e.g., via any suitable pixel to physical kinematics formula) the angular configuration of the robotic gripper 308 that makes centroids 2102 and 2104 have the same in-image pixel column coordinates (regardless of their in-image pixel row coordinates). Therefore, in this example, partial angular alignment (e.g., alignment in the yaw direction but not in the pitch direction) can be achieved, but not by making illumination spots 1304 and 1306 concentric, but rather by making their centroids have the same in-image pixel column coordinates.

[0243] As described above, the various embodiments described herein may involve a contour 2302 that is an ellipse defining both illumination spot 1304 and illumination spot 1306. In such cases, perfect angular alignment can be achieved by identifying (e.g., via any suitable pixel-to-physical kinematics formula) an angular configuration in which the ellipticity (e.g., the ratio of the major to minor radius) of contour 2302 is close to 1 (e.g., within any suitable threshold margin).

[0244] The scientific instrument systems, methods, or techniques disclosed herein may include interactions with human users (e.g., via references herein). Figure 39 The discussion pertains to the user's local computing device 3920. These interactions may include providing the user with information (e.g., about scientific instruments such as...). Figure 39Information on the operation of the scientific instrument (3910), information about the specimen being analyzed or other tests or measurements performed by the scientific instrument, information retrieved from local or remote databases or other information, or options for providing the user with input commands (e.g., controlling the scientific instrument (such as...) Figure 39 The operation of scientific instruments (3910), or control of the analysis, querying (e.g., querying local or remote databases), or other information generated by the scientific instruments. In some embodiments, these interactions can be performed via a graphical user interface (GUI), which includes a display device (e.g., referenced herein). Figure 38 A visual display on a display device 3810 (discussed herein) that provides output to a user and / or prompts the user to provide input (e.g., via reference herein). Figure 38 Other I / O devices discussed 3812 include one or more input devices, such as a keyboard, mouse, trackpad, or touchscreen. The scientific instrument systems, methods, or techniques disclosed herein may include any GUI suitable for user interaction.

[0245] Figure 37 Exemplary graphical user interface 3700 (hereinafter referred to as "GUI 3700") for performing some or all of the supporting methods or techniques disclosed herein, according to various embodiments, is depicted. In various aspects, GUI 3700 may be set up in a scientific instrument support system (e.g., as referenced herein). Figure 39 The computing device of the scientific instrument support system 3900 discussed herein (e.g., referenced herein) Figure 38 Any suitable electronic display (e.g., the one referred to herein) for the computing device 3800 discussed. Figure 38 The display device discussed is 3810), and the user or technician can use any suitable input device (e.g., the one referenced herein). Figure 38 The other I / O devices (3812) discussed and input technologies (e.g., cursor movement, motion capture, facial recognition, gesture detection, speech recognition, button activation) interact with the GUI 3700.

[0246] The GUI 3700 may include a data display area 3702, a data analysis area 3704, a scientific instrument control area 3706, and a settings area 3708. Figure 37 The specific number and arrangement of regions depicted are merely illustrative, and any number and arrangement of regions (including any desired features) can be included in other implementations of GUI 3700.

[0247] Data display area 3702 can display data generated by scientific instruments (e.g., as referenced in this article). Figure 39 The data generated by the scientific instrument 3910 discussed.

[0248] Data analysis area 3704 can display any suitable data analysis results (e.g., the results of analyzing data shown in data display area 3702 or other data). In some embodiments, data display area 3702 and data analysis area 3704 can be combined in GUI 3700 (e.g., including data output from scientific instruments and some analysis of the data in a public graphic or area).

[0249] The scientific instrument control area 3706 may include options that allow users or technicians to control the scientific instrument (e.g., as referenced herein). Figure 39 (Discussed scientific instrument 3910). For example, the scientific instrument control area 3706 may include configurable parameters for controlling the operation of such scientific instrument (e.g., configurable parameters for controlling the voltage or current of the scientific instrument, configurable parameters for controlling the internal temperature of the scientific instrument, or configurable parameters for controlling the fluid flow rate of the scientific instrument).

[0250] Setting up area 3708 may include options that allow a user or technician to control any feature or function of GUI 3700 (or other GUI) or perform common computational operations on data display area 3702 and data analysis area 3704 (e.g., storing data on a storage device, such as the one referenced herein). Figure 38 The storage device 3804 under discussion sends data to another user and tags the data.

[0251] As described above, the scientific instrument module 102 can be implemented by one or more computing devices. Figure 38 This is a block diagram of a computing device 3800 that can perform some or all of the scientific instrument methods or techniques disclosed herein, according to various embodiments. In some embodiments, the scientific instrument module 102 may be implemented by a single instance or multiple instances of the computing device 3800. Furthermore, as described below, the computing device 3800 (or multiple instances thereof) implementing the scientific instrument module 102 may be... Figure 39 A part of one or more of the scientific instrument 3910, the user local computing device 3920, the service local computing device 3930, or the remote computing device 3940.

[0252] Computing device 3800 is shown as having multiple components, but any one or more of these components may be omitted or duplicated depending on the application and setup. In some embodiments, some or all of the components included in computing device 3800 may be attached to one or more motherboards and encapsulated in a housing (e.g., including plastic, metal, or other materials). In some embodiments, some of these components may be fabricated on a single system-on-a-chip (SoC) (e.g., the SoC may include one or more instances of processing device 3802 and one or more instances of storage device 3804). Additionally, in various embodiments, computing device 3800 may be omitted. Figure 38 One or more of the components shown may be included, but may include interface circuitry (not shown) for coupling to one or more omitted components using any suitable interface (e.g., Universal Serial Bus (USB) interface, High Definition Multimedia Interface (HDMI) interface, Controller Area Network (CAN) interface, Serial Peripheral Interface (SPI) interface, Ethernet interface, wireless interface, or any other suitable interface). For example, computing device 3800 may omit display device 3810, but may include display device interface circuitry (e.g., connector and driver circuitry) to which display device 3810 may be coupled.

[0253] Computing device 3800 may include processing device 3802 (e.g., one or more processing devices). As used herein, the term "processing device" can refer to any device or part of a device that processes electronic data from a register or memory to convert that electronic data into other electronic data that can be stored in the register or memory. Processing device 3802 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms within hardware), server processors, or any other suitable processing device.

[0254] Computing device 3800 may include storage device 3804 (e.g., one or more storage devices). Storage device 3804 may include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive bridged RAM (CBRAM) devices), hard disk drive-based memory devices, solid-state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, storage device 3804 may include memory sharing a die with processing device 3802. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 3804 may include a non-transitory computer-readable medium having instructions thereon that, when executed by one or more processing devices (e.g., processing device 3802), cause computing device 3800 to perform any suitable method or portion thereof disclosed herein.

[0255] Computing device 3800 may include interface device 3806 (e.g., one or more instances of interface device 3806). Interface device 3806 may include one or more communication chips, connectors, or other hardware and software to manage communication between computing device 3800 and other computing devices. For example, interface device 3806 may include circuitry for managing wireless communication used to transmit data to and from computing device 3800. The term "wireless" and its derivatives can be used to describe circuits, devices, systems, methods, techniques, or communication channels that can transmit data over a non-solid medium using modulated electromagnetic radiation. This term does not imply that the associated device does not contain any wires, although in some embodiments it may not contain any wires. The circuitry included in interface device 3806 for managing wireless communications can implement any of a variety of wireless standards or protocols, including but not limited to Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.11 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendments), Long Term Evolution (LTE) projects, and any amendments, updates, and / or revisions (e.g., Advanced LTE project, Ultra Mobile Broadband (UMB) project (also known as “3GPP2”)). In some implementations, the circuitry included in interface device 3806 for managing wireless communications may operate according to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE networks. In some embodiments, the circuitry included in interface device 3806 for managing wireless communications may operate according to Enhanced Data GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in interface device 3806 for managing wireless communications may operate according to Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolved Data Optimization (EV-DO) and its derivatives, as well as any other wireless protocol designated as 3G, 4G, 5G, or higher. In some embodiments, interface device 3806 may include one or more antennas (e.g., one or more antenna arrays) to receive and / or transmit wireless communications.

[0256] In some embodiments, interface device 3806 may include circuitry for managing wired communications, such as electrical communication protocols, optical communication protocols, or any other suitable communication protocol. For example, interface device 3806 may include circuitry supporting communications based on Ethernet technology. In some embodiments, interface device 3806 may support both wireless and wired communications, or it may support multiple wired communication protocols or multiple wireless communication protocols. For example, a first set of circuitry for interface device 3806 may be dedicated to short-range wireless communications such as Wi-Fi or Bluetooth, and a second set of circuitry for interface device 3806 may be dedicated to long-range wireless communications such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, a first set of circuitry for interface device 3806 may be dedicated to wireless communications, and a second set of circuitry for interface device 3806 may be dedicated to wired communications.

[0257] The computing device 3800 may include a battery / power circuit 3808. The battery / power circuit 3808 may include one or more energy storage devices (e.g., batteries or capacitors) or circuitry for coupling components of the computing device 3800 to a power source (e.g., AC line power) separate from the computing device 3800.

[0258] The computing device 3800 may include a display device 3810 (e.g., multiple display devices). The display device 3810 may include any visual indicator, such as a head-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0259] The computing device 3800 may include other input / output (I / O) devices 3812. For example, other I / O devices 3812 may include one or more audio output devices (e.g., speakers, headphones, earphones, alarms), one or more audio input devices (e.g., microphones or microphone arrays), positioning devices (e.g., GPS devices that communicate with a satellite-based system to receive the location of the computing device 3800), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes), image capture devices such as cameras, keyboards, cursor control devices such as mice, styluses, trackballs or touchpads, barcode readers, quick-response (QR) code readers, or radio frequency identification (RFID) readers.

[0260] The computing device 3800 may have any suitable form factor for its application and setup, such as a handheld or mobile computing device (e.g., a mobile phone, smartphone, mobile internet device, tablet computer, laptop computer, netbook computer, ultrabook computer, personal digital assistant (PDA), ultra-mobile personal computer), desktop computing device, or server computing device or other networked computing component.

[0261] One or more computing devices that implement any scientific instrument module, method or technique disclosed herein may be part of a scientific instrument support system. Figure 39 This is a block diagram of an exemplary scientific instrument support system 3900 according to various implementation schemes, in which some or all of the scientific instrument support methods disclosed herein can be performed. The scientific instrument modules, methods, or techniques disclosed herein (e.g., scientific instrument module 102, computer-implemented method 200, system 322, computer-implemented methods 2700 to 3000) can be implemented by one or more scientific instruments 3910, user local computing devices 3920, service local computing devices 3930, or remote computing devices 3940 of the scientific instrument support system 3900.

[0262] Any device in the scientific instrument 3910, user local computing device 3920, service local computing device 3930, or remote computing device 3940 may include any implementation of computing device 3800, and any device in the scientific instrument 3910, user local computing device 3920, service local computing device 3930, or remote computing device 3940 may take the form of any suitable implementation of computing device 3800.

[0263] Scientific instrument 3910, user local computing device 3920, service local computing device 3930, or remote computing device 3940 may each include a processing device 3902, a storage device 3904, and an interface device 3906. The processing device 3902 may take any suitable form, including any form of processing device 3802, and the processing devices 3902 included in different devices of scientific instrument 3910, user local computing device 3920, service local computing device 3930, or remote computing device 3940 may take the same or different forms. The storage device 3904 may take any suitable form, including any form of storage device 3804, and the storage devices 3904 included in different devices of scientific instrument 3910, user local computing device 3920, service local computing device 3930, or remote computing device 3940 may take the same or different forms. Interface device 3906 may take any suitable form, including any form of interface device 3806, and the interface device 3906 included in different devices such as scientific instrument 3910, user local computing device 3920, service local computing device 3930 or remote computing device 3940 may take the same or different forms.

[0264] Scientific instrument 3910, user local computing device 3920, service local computing device 3930, and remote computing device 3940 can communicate with other elements of scientific instrument support system 3900 via communication path 3908. Communication path 3908 can communicatively couple interface devices 3906 of different elements among the elements of scientific instrument support system 3900, as shown, and can be a wired or wireless communication path (e.g., any communication technology discussed herein with reference to interface device 3806). Figure 39 The specific scientific instrument support system 3900 depicted includes communication paths between each pair of devices among scientific instrument 3910, user local computing device 3920, service local computing device 3930, and remote computing device 3940. However, this "fully connected" implementation is merely illustrative, and in various embodiments, various communication paths in communication path 3908 may not exist. For example, in some embodiments, service local computing device 3930 may lack a direct communication path 3908 between its interface device 3906 and the interface device 3906 of scientific instrument 3910, but may instead communicate with scientific instrument 3910 via communication paths 3908 between service local computing device 3930 and user local computing device 3920 and between user local computing device 3920 and scientific instrument 3910.

[0265] Scientific instrument 3910 may include any suitable scientific instrument, such as charged particle microscope 302.

[0266] User-local computing device 3920 may be a user-local computing device of scientific instrument 3910 (e.g., according to any embodiment of computing device 3800). In some embodiments, user-local computing device 3920 may also be local to scientific instrument 3910, but this is not necessarily the case; for example, user-local computing device 3920 located in a user's home or office may be remote from scientific instrument 3910 but communicate with it, allowing the user to use user-local computing device 3920 to control and / or access data from scientific instrument 3910. In some embodiments, user-local computing device 3920 may be a laptop, smartphone, or tablet device. In some embodiments, user-local computing device 3920 may be a portable computing device.

[0267] The servicing local computing device 3930 may be a computing device physically local to the scientific instrument 3910 (e.g., according to any embodiment of computing device 3800). For example, the servicing local computing device 3930 may be local to the manufacturer of the scientific instrument 3910 or a third-party service company. In some embodiments, the servicing local computing device 3930 may communicate with the scientific instrument 3910, the user local computing device 3920, or the remote computing device 3940 (e.g., via a direct communication path 3908 or via multiple “indirect” communication paths 3908, as described above) to receive data regarding the operation of the scientific instrument 3910, the user local computing device 3920, or the remote computing device 3940 (e.g., self-test results of the scientific instrument 3910, calibration coefficients used by the scientific instrument 3910, and measurements from sensors associated with the scientific instrument 3910). In some implementations, the service local computing device 3930 can communicate with the scientific instrument 3910, the user local computing device 3920, or the remote computing device 3940 (e.g., via a direct communication path 3908 or via multiple "indirect" communication paths 3908, as described above) to transfer data to the scientific instrument 3910, the user local computing device 3920, or the remote computing device 3940 (e.g., thereby updating programming instructions (such as firmware) in the scientific instrument 3910, initiating the execution of test or calibration sequences in the scientific instrument 3910, and updating programming instructions (such as software) in the user local computing device 3920 or the remote computing device 3940). Users of the scientific instrument 3910 can use the scientific instrument 3910 or the user local computing device 3920 to communicate with the service local computing device 3930 to report problems with the scientific instrument 3910 or the user local computing device 3920, request a technician visit to improve the operation of the scientific instrument 3910, order consumables or replacement parts associated with the scientific instrument 3910, or for other purposes.

[0268] Remote computing device 3940 may be a computing device located remotely from scientific instrument 3910 or user local computing device 3920 (e.g., any embodiment of computing device 3800 discussed herein). In some embodiments, remote computing device 3940 may be included in a data center or other large-scale server environment. In some embodiments, remote computing device 3940 may include network-attached storage (e.g., as part of storage device 3904). Remote computing device 3940 may store data generated by scientific instrument 3910, perform analysis of data generated by scientific instrument 3910 (e.g., according to programming instructions), facilitate communication between user local computing device 3920 and scientific instrument 3910, or facilitate communication between service local computing device 3930 and scientific instrument 3910.

[0269] In some implementation schemes, this can be omitted. Figure 39 One or more components of the scientific instrument support system 3900 shown. Furthermore, in some embodiments, there may be... Figure 39 The scientific instrument support system 3900 comprises multiple components among various elements. For example, the scientific instrument support system 3900 may include multiple user local computing devices 3920 (e.g., different user local computing devices 3920 associated with different users or located in different locations). In another example, the scientific instrument support system 3900 may include multiple scientific instruments 3910, all of which communicate with a service local computing device 3930 and / or a remote computing device 3940; in this implementation, the service local computing device 3930 may monitor these multiple scientific instruments 3910, and the service local computing device 3930 may cause updates or other information to be simultaneously “broadcast” to multiple scientific instruments 3910. The different scientific instruments 3910 in the scientific instrument support system 3900 may be close to each other (e.g., in the same room) or far from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some implementations, scientific instrument 3910 can be connected to an Internet of Things (IoT) stack that allows command and control of scientific instrument 3910 via web-based applications, virtual or augmented reality applications, mobile applications, or desktop applications. Any of these applications can be accessed by a user operating a user-local computing device 3920 that communicates with scientific instrument 3910 via an intermediate remote computing device 3940. In some implementations, scientific instrument 3910 may be sold by the manufacturer along with one or more associated user-local computing devices 3920 that are part of a local scientific instrument computing unit 3912.

[0270] In some embodiments, the different scientific instruments 3910 included in the scientific instrument support system 3900 can be of different types; for example, one scientific instrument 3910 can be a mass spectrometer, while another scientific instrument 3910 can be a chromatograph or an autosampler. In some such embodiments, a remote computing device 3940 or a user-local computing device 3920 can combine data from the different types of scientific instruments 3910 included in the scientific instrument support system 3900.

[0271] In various instances, machine learning algorithms or models can be implemented in any suitable manner to facilitate any suitable aspect described herein. To facilitate some of the machine learning aspects described above in various implementations, consider the following discussion of artificial intelligence (AI). The various implementations described herein can employ artificial intelligence to facilitate the automation of one or more features or functionalities. These components can employ various AI-based schemes to perform the various implementations / examples disclosed herein. To provide or assist in the numerous decisions described herein (e.g., decision, ascertainment, inference, computation, prediction, prognosis, estimation, derivation, forecasting, detection, computation), the components described herein can examine all or a subset of the data to which they have been granted access and can provide reasoning or determination of the state of a system or environment from a set of observations, such as those captured via events or data. For example, decisions can be employed to identify specific contexts or actions, or a probability distribution of states can be generated. These decisions can be probabilistic; that is, the probability distribution of states of interest is computed based on considerations of data and events. Decision can also refer to techniques used to compose higher-level events from a set of events or data.

[0272] Such judgments can lead to the construction of new events or actions from a set of observed events or stored event data, regardless of whether the events are closely related in time or whether the events and data come from one or more event and data sources. The components disclosed herein can employ various classification schemes (explicit training (e.g., via training data) and implicit training (e.g., via observed behavior, preferences, historical information, received external information, etc.)) or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) related to the execution of automatic or judgmental actions relevant to the claimed subject matter. Therefore, classification schemes or systems can be used to automatically learn and execute multiple functions, actions, or judgments.

[0273] The classifier can take the input attribute vector z = (z1, z2, z3, z4, z5) as input. nThe classification is mapped to the confidence level of the input belonging to a certain class, such as f(z) = confidence level (class). This classification can use probabilistic or statistical analysis (e.g., considering the utility and cost of analysis) to determine the action to be automatically performed. Support Vector Machines (SVMs) can be used as an example of a classifier that can be adopted. SVMs operate by finding a hypersurface in the space of possible inputs, where the hypersurface attempts to separate triggering criteria from non-triggering events. Intuitively, this makes the classification correct for test data that is close to but not identical to the training data. Other directed and non-directed model classification methods include, for example, Naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide different independent patterns, any of which can be adopted. The classification used in this paper also includes statistical regression for developing priority models.

[0274] To provide additional context for the various implementation schemes described herein, Figure 40 The following discussion is intended to provide a brief, general description of a suitable computing environment 4000 in which various implementations of the embodiments described herein may be carried out. Although the embodiments have been described above in the general context of computer-executable instructions that can run on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules or as a combination of hardware and software.

[0275] Typically, program modules include routines, programs, components, data structures, etc., that perform specific tasks or implement specific abstract data types. Furthermore, those skilled in the art will understand that the methods of this invention can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, and personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, each operatively coupled to one or more associated devices.

[0276] The implementation schemes shown in this paper can also be practiced in distributed computing environments, where some tasks are performed by remote processing devices linked via a communication network. In a distributed computing environment, program modules can reside on both local and remote memory storage devices.

[0277] Computing devices typically include a variety of media, which may include computer-readable storage media, machine-readable storage media, or communication media. These two terms are used interchangeably herein, as follows. A computer-readable storage media or a machine-readable storage media can be any available storage medium accessible by a computer and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, a computer-readable storage media or a machine-readable storage media can be implemented in conjunction with any method or technology used for storing information, such as computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.

[0278] Computer-readable storage media may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compressed optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), Blu-ray disc (BD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, solid-state drives or other solid-state storage devices, or other tangible or non-transitory media that can be used to store desired information. In this regard, the terms “tangible” or “non-transitory” used herein to describe storage devices, memories, or computer-readable media should be understood to exclude only the propagation of transient signals themselves as a modifier, and do not waive the rights of all standard storage devices, memories, or computer-readable media that do not merely propagate transient signals themselves.

[0279] Computer-readable storage media can be accessed by one or more local or remote computing devices, for example via access requests, queries or other data retrieval protocols, for various operations concerning the information stored on the media.

[0280] Communication media typically contain computer-readable instructions, data structures, program modules, or other structured or unstructured data in data signals (such as modulated data signals, e.g., carrier waves or other transmission mechanisms), and include any information transmission or delivery medium. The term "modulated data signal" or signal refers to a signal whose characteristics are set or altered to encode information in one or more signals. By way of example (but not limited to), communication media include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media).

[0281] Refer again Figure 40An exemplary environment 4000 for implementing various embodiments of the aspects described herein includes a computer 4002, which includes a processing unit 4004, system memory 4006, and a system bus 4008. The system bus 4008 couples system components, including but not limited to system memory 4006, to the processing unit 4004. The processing unit 4004 can be any of a variety of commercially available processors. Dual microprocessors and other multiprocessor architectures may also be used as the processing unit 4004.

[0282] System bus 4008 can be any of several types of bus architectures, and can further interconnect with memory buses (with or without memory controllers), peripheral buses, and local buses using any of a variety of commercially available bus architectures. System memory 4006 includes ROM 4010 and RAM 4012. The Basic Input / Output System (BIOS) can be stored in non-volatile memory such as ROM, erasable programmable read-only memory (EPROM), or EEPROM, where the BIOS contains basic routines such as those that facilitate the transfer of information between components within computer 4002 during startup. RAM 4012 may also include high-speed RAM, such as static RAM for caching data.

[0283] Computer 4002 also includes an internal hard disk drive (HDD) 4014 (e.g., EIDE, SATA) and may include one or more external storage devices 4016 (e.g., floppy disk drive (FDD) 4016, memory stick or flash drive reader, memory card reader, etc.) and drives 4020, such as solid-state drives, optical disc drives, which can read from or write to disks 4022 (e.g., CD-ROM, DVD, BD, etc.). Alternatively, if a solid-state drive is involved, disk 4022 is not included unless provided separately. Although the internal HDD 4014 is shown as residing within computer 4002, the internal HDD 4014 may also be configured for external use in a suitable chassis (not shown). Additionally, although not shown in environment 4000, solid-state drives (SSDs) may be used to supplement or replace HDD 4014. HDD 4014, external storage device 4016, and drive 4020 can be connected to system bus 4008 via HDD interface 4024, external storage interface 4026, and drive interface 4028, respectively. Interface 4024 for the specific implementation of the external drive may include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are also considered in the embodiments described herein.

[0284] Drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, etc. For computer 4002, drives and storage media accommodate storage of any data in a suitable digital format. Although the above description of computer-readable storage media refers to a corresponding type of storage device, those skilled in the art will understand that other types of computer-readable storage media (whether currently existing or developed in the future) can also be used in the exemplary operating environment, and further, any such storage media may contain computer-executable instructions for performing the methods described herein.

[0285] The driver and RAM 4012 can store multiple program modules, including an operating system 4030, one or more application programs 4032, other program modules 4034, and program data 4036. All or part of the operating system, applications, modules, or data can also be cached in RAM 4012. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.

[0286] Computer 4002 may optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate the hardware environment used for operating system 4030, and the emulated hardware may optionally be different from that of the operating system 4030. Figure 40 The hardware shown is illustrated. In such implementations, the operating system 4030 may include one of a plurality of virtual machines (VMs) hosted at the computer 4002. Furthermore, the operating system 4030 may provide a runtime environment for the application 4032, such as the Java Runtime Environment or the .NET Framework. A runtime environment is a consistent execution environment that allows the application 4032 to run on any operating system that includes a runtime environment. Similarly, the operating system 4030 may support containers, and the application 4032 may be in the form of a container, which is a lightweight, standalone, executable software package that includes, for example, the application's code, runtime environment, system tools, system libraries, and settings.

[0287] Furthermore, computer 4002 may be equipped with security modules, such as a Trusted Processing Module (TPM). For example, using a TPM, the boot component hashes the next boot component periodically and waits for the result to match a security value before loading the next boot component. This process can occur at any layer of the computer 4002's code execution stack, for example, at the application execution level or the operating system (OS) kernel level, thereby achieving security at any level of code execution.

[0288] Users can input commands and information into computer 4002 through one or more wired / wireless input devices (e.g., keyboard 4038, touchscreen 4040, and pointing devices such as mouse 4042). Other input devices (not shown) may include microphones, infrared (IR) remote controls, radio frequency (RF) remote controls or other remote controls, joysticks, virtual reality controllers or virtual reality headsets, game controllers, styluses, image input devices (e.g., cameras), gesture sensor input devices, visual motion sensor input devices, emotion or face detection devices, biometric input devices (e.g., fingerprint or iris scanners), etc. These and other input devices are typically connected to processing unit 4004 via input device interface 4044, which can be coupled to system bus 4008, but may also be connected via other interfaces such as parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, etc. Interfaces, etc.

[0289] Monitor 4046 or other types of display devices can also be connected to system bus 4008 via an interface such as video adapter 4048. In addition to monitor 4046, the computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.

[0290] Computer 4002 can operate in a networked environment using logical connections to one or more remote computers (such as remote computer 4050) via wired or wireless communications. Remote computer 4050 can be a workstation, server computer, router, personal computer, laptop computer, microprocessor-based entertainment device, peer-to-peer device, or other common network node, and typically includes many or all of the elements described relative to computer 4002, but for brevity only memory / storage device 4052 is shown. The depicted logical connections include wired / wireless connections to a local area network (LAN) 4054 or a larger network (e.g., a wide area network (WAN) 4056). Such LAN and WAN networking environments are common in offices and companies and facilitate the establishment of enterprise-wide computer networks (e.g., intranets), all of which can connect to global communications networks (e.g., the Internet).

[0291] When used in a LAN networking environment, computer 4002 can connect to local network 4054 via a wired or wireless communication network interface or adapter 4058. Adapter 4058 facilitates wired or wireless communication with LAN 4054, which may also include a wireless access point (AP) configured thereon for wireless communication with adapter 4058.

[0292] When used in a WAN networking environment, computer 4002 may include modem 4060 or a communication server that can be otherwise connected to WAN 4056 to establish communication via WAN 4056, such as via the Internet. Modem 4060 may be built-in or external, wired or wireless, and may be connected to system bus 4008 via input device interface 4044. In a network environment, program modules shown relative to computer 4002 or parts thereof may be stored in remote memory / storage device 4052. It should be understood that the network connections shown are exemplary and other components for establishing communication links between computers may be used.

[0293] When used in a LAN or WAN networking environment, computer 4002 can access cloud storage systems or other network-based storage systems, such as, but not limited to, network virtual machines that provide one or more aspects of information storage or processing, in addition to or as an alternative to external storage device 4016 as described above. Generally, the connection between computer 4002 and the cloud storage system can be established via LAN 4054 or WAN 4056, for example, via adapter 4058 or modem 4060, respectively. When computer 4002 is connected to an associated cloud storage system, external storage interface 4026 can manage the storage provided by the cloud storage system with the help of adapter 4058 or modem 4060, just as it would manage other types of external storage. For example, external storage interface 4026 can be configured to provide access to cloud storage sources as if those sources were physically connected to computer 4002.

[0294] Computer 4002 is operable to communicate with any wireless device or entity operating wirelessly (e.g., printer, scanner, desktop or portable computer, portable data assistant, communications satellite, any device or location associated with a wirelessly detectable tag (e.g., kiosks, newsstands, store shelves, etc.) and telephone). This may include Wi-Fi and Wireless technology. Therefore, communication can be a predefined structure like a traditional network, or it can be temporary communication between at least two devices.

[0295] Figure 41This is a schematic block diagram of an exemplary computing environment 4100 with which the disclosed subject matter can interact. The exemplary computing environment 4100 includes one or more clients 4110. Client 4110 may be hardware or software (e.g., threads, processes, computing devices). The exemplary computing environment 4100 also includes one or more servers 4130. Server 4130 may also be hardware or software (e.g., threads, processes, computing devices). For example, server 4130 may accommodate threads to perform transformations by employing one or more embodiments described herein. One possible communication between client 4110 and server 4130 may take the form of data packets suitable for transmission between two or more computer processes. The exemplary computing environment 4100 includes a communication framework 4150 that can be used to facilitate communication between client 4110 and server 4130. Client 4110 is operatively connected to one or more client data repositories 4120 that can be used to store information local to client 4110. Similarly, server 4130 is operatively connected to one or more server data repositories 4140 that can be used to store information local to server 4130.

[0296] Exemplary non-limiting apparatuses for carrying out the various embodiments described herein are shown below. Figure 42 . Figure 42 A non-limiting example of a dual-beam system 4210 is shown, having a vertically mounted scanning electron microscope (SEM) tube and a focused ion beam (FIB) tube mounted at an angle of approximately 52 degrees from the vertical direction. Such a dual-beam system can be commercially available, for example, from FEI Company, Hillsboro, Oregon, the assignee of this application. However... Figure 42 Examples of suitable microscopy hardware that can be used to implement the various embodiments described herein are shown; however, it should be understood that such microscopy hardware is non-limiting. In other words, the various embodiments described herein can be used in conjunction with any other suitable type of microscope hardware. The dual-beam system 4210 is a non-limiting example of the charged particle microscope 302 or any other scientific instrument described above.

[0297] The dual-beam system 4210 can be equipped with a scanning electron microscope 4241 and a power supply and control unit 4245. An electron beam 4243 is emitted from the cathode 4252 by applying a voltage between the cathode 4252 and the anode 4254. The electron beam 4243 can be focused into a fine point by a condenser lens 4256 and an objective lens 4258. The electron beam 4243 can be used for two-dimensional scanning of any suitable specimen by means of a deflection coil 4260. The operation of the condenser lens 4256, the objective lens 4258, or the deflection coil 4260 can be controlled by the power supply and control unit 4245.

[0298] An electron beam 4243 can be focused onto a substrate 4222, which is located on a movable XY stage 4225 within a lower chamber 4226. When electrons in the electron beam 4243 strike the substrate 4222, secondary electrons are emitted. These secondary electrons can be detected by a secondary electron detector 4240, as described below. A STEM detector 4262, located below the scanning transmission electron microscope (STEM) sample holder 4224 and the movable XY stage 4225, can collect electrons passing through the sample mounted on the STEM sample holder 4224, as described above.

[0299] The dual-beam system 4210 may also include a focused ion beam (FIB) system 4211, which may include a vacuum chamber having an upper neck 4212 within which an ion source 4214 and a focusing column 4216 may be located. The focusing column includes extraction electrodes and an electrostatic optics system (in some cases, the upper neck may also be referred to as the ion column 4212). The axis of the focusing column 4216 may be tilted 52 degrees (or any other suitable angular displacement) relative to the axis of the electron column. The ion column 4212 may include an ion source 4214, extraction electrodes 4215, a focusing element 4217, a deflection element 4220, and a focused ion beam 4218. The focused ion beam 4218 may pass from the ion source 4214 through the focusing column 4216 and propagate toward a substrate 4222 between electrostatic deflection tools schematically indicated by reference numeral 4220. The substrate may include, for example, a semiconductor device positioned on a movable XY stage 4225 within a lower chamber 4226.

[0300] The movable XY stage 4225 can move in the horizontal plane (along the X and Y axes) and the vertical plane (along the Z axis). The movable XY stage 4225 can tilt approximately sixty (60) degrees and rotate about the Z axis. In some embodiments, a separate STEM sample stage (not shown) may be used. Such a STEM sample stage can also move along the X, Y, and Z axes. The door 4261 is opened for inserting the substrate 4222 onto the movable XY stage 4225, or for maintaining the internal gas supply reservoir (if used). The door 4261 can be interlocked so that it cannot be opened if the system is under vacuum.

[0301] An ion pump 4268 can be used to evacuate the neck section 4212. Under the control of a vacuum controller 4232, the chamber 4226 is evacuated using a turbomolecular and mechanical pumping system 4230. This type of vacuum system provides a vacuum level within the chamber 4226 of approximately 1 x 10⁻⁶. -7 5x10 -4 The vacuum between the chambers. If an etching assist gas, etching delay gas, or deposition precursor gas is used, the chamber background pressure can be increased, typically to about 1 x 10⁻⁶. -5 Entrust.

[0302] A high-voltage power supply 4234 provides an appropriate accelerating voltage to the electrodes in the focusing column 4216 to excite and focus the ion beam 4218. When it strikes the substrate 4222, material is sputtered (i.e., physically ejected) from the sample. Alternatively, the focused ion beam 4218 can decompose a precursor gas to deposit material.

[0303] A high-voltage power supply 4234 can be connected to an ion source 4214 (which may be a liquid metal ion source) and appropriate electrodes in the ion beam focusing column 4216 to form an ion beam 4218 of approximately 1 keV to 60 keV and guide it toward the sample. A deflection controller and amplifier 4236, operating according to a pattern provided by a pattern generator 4238, can be coupled to a deflection element 4220 (which may be a deflection plate), thereby allowing manual or automatic control of the focused ion beam 4218 to draw a corresponding pattern on the upper surface of the substrate 4222. In some systems, the deflection element 4220 may be placed in front of the final lens. When a blanking controller (not shown) applies a blanking voltage to the blanking electrode, the blanking electrode (not shown) within the ion beam focusing column 4216 causes the focused ion beam 4218 to strike the blanking aperture (not shown) instead of the substrate 4222.

[0304] Ion source 4214 can provide a metal ion beam, such as gallium. In other examples, ion source 4214 can be a plasma ion source, extracting ions from the generated plasma. This source can be focused into a beam sub-1 / 10 of a micrometer wide at substrate 4222 for modifying substrate 4222 by ion milling, enhanced etching, material deposition, or for imaging substrate 4222.

[0305] A charged particle detector 4240, such as an Everhart Thornley or multichannel board, for detecting secondary ion or electron emissions, can be connected to video circuitry 4242, which supplies drive signals to video monitor 4244 and receives deflection signals from system controller 4219. The position of the charged particle detector 4240 within the lower chamber 4226 can vary in different embodiments. For example, the charged particle detector 4240 can be coaxial with the ion beam and include apertures for allowing the ion beam to pass through. In other embodiments, secondary particles may be collected by a final lens and then deflected off-axis for collection.

[0306] Micromanipulator 4247 can precisely move an object within a vacuum chamber. Micromanipulator 4247 may include a precision electric motor 4248 positioned outside the vacuum chamber to provide X, Y, Z, and θ control of a portion 4249 positioned within the vacuum chamber. Micromanipulator 4247 may be equipped with different end effectors for manipulating small objects. In the various embodiments described herein, the end effector may be a fine probe 4250.

[0307] Gas delivery system 4246 may extend into lower chamber 4226 for introducing and guiding gaseous vapor into substrate 4222. A suitable gas delivery system 4246 is described in U.S. Patent No. 5,851,413, "Gas Delivery Systems for Particle Beam Processing" by Casella et al. (assigned to the assignee of this invention). Another gas delivery system is described in U.S. Patent No. 5,435,850, "Gas Injection System" by Rasmussen (also assigned to the assignee of this invention). For example, iodine may be delivered to enhance etching, or organometallic compounds may be delivered to deposit metal.

[0308] System controller 4219 controls the operation of various parts of dual-beam system 4210. Through system controller 4219, a user can scan the focused ion beam 4218 or electron beam 4243 in a desired manner by inputting commands into any suitable user interface (not shown). Alternatively, system controller 4219 can control dual-beam system 4210 according to programming instructions stored in memory 4221. In various embodiments, any one of one or more software components 327 may be implemented in system controller 4219 or otherwise executed by system controller.

[0309] Various implementations can be systems, methods, apparatus, or computer program products at any possible level of technical detail integration. Computer program products may include computer-readable storage media (or media) having computer-readable program instructions for causing a processor to execute aspects of various implementations. Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media may also include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards or raised structures in grooves on which instructions are recorded), and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses transmitted through fiber optic cables), or electrical signals transmitted through wires.

[0310] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network) to an external computer or external storage device. This network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device. The computer-readable program instructions used to perform operations in various implementation schemes may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). Computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider). In some implementations, the electronic circuitry includes, for example, programmable logic circuitry, a field-programmable gate array (FPGA), or a programmable logic array (PLA), which can be personalized by utilizing the state information of the computer-readable program instructions to execute the computer-readable program instructions for various purposes.

[0311] This document describes various aspects with reference to flowchart illustrations or block diagrams of methods, apparatus (systems), and computer program products according to various embodiments. It should be understood that each block in the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create components for implementing the functions / actions specified in one or more blocks of the flowchart or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, or other device to operate in a particular manner. The computer-readable storage medium storing the instructions includes an article of manufacture comprising instructions that implement aspects of the functions / actions specified in one or more blocks of the flowchart or block diagram. The computer-readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational actions to be performed on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / actions specified in one or more blocks of the flowchart or block diagram.

[0312] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each box in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions marked in the boxes may occur in a different order than indicated in the figures. For example, two consecutively displayed boxes may actually be executed substantially simultaneously, or sometimes in reverse order, depending on the functions involved. It should also be noted that each box in a block diagram or flowchart, and combinations of boxes in a block diagram or flowchart, may be implemented by a dedicated hardware system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0313] Although the subject matter has been described above in the general context of computer-executable instructions for computer program products already running on a computer, those skilled in the art will recognize that this disclosure can also be implemented, or can be combined with other program modules. Typically, program modules include routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, those skilled in the art will recognize that various aspects can be practiced using other computer system configurations, including single-processor or multi-processor computer systems, small computing devices, mainframe computers, and computers, handheld computing devices (e.g., PDAs, telephones), microprocessor-based or programmable consumer or industrial electronics, etc. The aspects shown can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected via a communication network. However, some (if not all) aspects of this disclosure can be practiced on a standalone computer. In a distributed computing environment, program modules can reside in both local memory storage devices and remote memory storage devices.

[0314] The terms “component,” “system,” “platform,” “interface,” etc., used in this application may refer to or include computer-related entities or entities associated with an operating machine having one or more specific functions. Entities disclosed herein may be hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, processor, object, executable program, executing thread, program, or computer running on a processor. As an example, an application running on a server and the server itself can both be components. One or more components may reside within a process or executing thread, and components may reside on a single computer or be distributed across two or more computers. In another example, a corresponding component may be executed from various computer-readable media on which various data structures are stored. These components may communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from one component interacts with another component in a local system, a distributed system, or with other systems via a network such as the Internet). As another example, a component may be a device having specific functionality provided by mechanical parts operated by electrical or electronic circuitry, which is operated by a software or firmware application executed by a processor. In this case, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application. As yet another example, a component can be a device that provides specific functionality through electronic components without mechanical parts, wherein the electronic components may include a processor or other components for executing software or firmware that at least partially endows the electronic components with functionality. In one aspect, a component can be simulated via a virtual machine, for example, within a cloud computing system.

[0315] Furthermore, the term “or” is intended to mean inclusive “or” rather than exclusive “or”. That is, unless otherwise specified or clearly apparent from the context, “X adopts A or B” is intended to mean any natural inclusive permutation. That is, if X adopts A; X adopts B; or X adopts both A and B, then “X adopts A or B” is satisfied in any of the foregoing instances. As used herein, the term “and / or” is intended to have the same meaning as “or”. Furthermore, unless otherwise specified or clearly apparent from the context involving the singular form, the article “a” as used in the subject matter description and accompanying drawings should generally be interpreted as meaning “one or more”. As used herein, the terms “example” or “exemplary” are used to indicate as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited to such examples. Furthermore, any aspect or design described herein as “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor does it exclude equivalent exemplary structures and techniques known to those skilled in the art.

[0316] The disclosure herein describes non-limiting examples. For ease of description or explanation, the various sections disclosed herein use the terms "each," "every," or "all" when discussing various examples. The use of terms such as "each," "every," or "all" is not restrictive. In other words, when the disclosure herein provides a description of "each," "every," or "all" applicable to a particular object or component, it should be understood that this is only a non-limiting example, and it should also be understood that in various other examples, such a description may apply to fewer descriptions of "each," "every," or "all" for less than that particular object or component.

[0317] As used in this subject matter specification, the term "processor" can substantially refer to any computing processing unit or device, including but not limited to a single-core processor; a single processor with software multithreading capabilities; a multi-core processor; a multi-core processor with software multithreading capabilities; a multi-core processor with hardware multithreading technology; a parallel platform; and a parallel platform with distributed shared memory. Additionally, a processor can refer to an integrated circuit, application-specific integrated circuit (ASIC), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic controller (PLC), complex programmable logic device (CPLD), discrete gate or transistor logic components, discrete hardware components, or any combination thereof, designed to perform the functions described herein. Furthermore, processors can utilize nanoscale architectures, such as (but not limited to) molecular and quantum dot-based transistors, switches, and gates, to optimize space utilization or enhance the performance of user devices. Processors can also be implemented as a combination of computing processing units. In this disclosure, terms such as "repository," "storage device," "data repository," "data storage device," "database," and any other information storage component substantially related to the operation and functionality of a component are used to refer to a "memory component," an entity embodied in "memory," or a component that includes memory. It should be understood that memory and / or memory components can be volatile or non-volatile memory, or may include both. By way of illustration and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory may include RAM, which may, for example, serve as external cache memory. By way of illustration and not limitation, RAM takes many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous linked DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Furthermore, the memory components of the systems or computer-implemented methods disclosed herein are intended to include, but are not limited to, these and any other suitable types of memory.

[0318] The foregoing only includes examples of systems and computer-implemented methods. It is certainly impossible to describe every conceivable combination of components or computer-implemented methods for the purposes of describing this disclosure, but many further combinations and arrangements of the contents of this disclosure are possible. Furthermore, with regard to the use of the terms “comprising,” “having,” “possessing,” etc., in the detailed description, claims, appendices, and drawings, these terms are intended to be inclusive in a manner similar to how the term “comprising” is interpreted when used as a transitional word in a claim.

[0319] Various embodiments have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the embodiments disclosed herein. Many modifications and variations are obvious without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements relative to the technology on the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0320] The following examples illustrate the various non-restrictive aspects.

[0321] Example 1: An apparatus may include: a charged particle microscope having a vacuum chamber, wherein the vacuum chamber may include a robotic gripper and a microscope grid container; and a processor configured to teach the robotic gripper to align its axis-to-axis with the aperture-to-axis of the microscope grid container angularly or translationally based on a light source coupled to the robotic gripper.

[0322] Example 2: An apparatus that can implement any of the foregoing examples, wherein the microscope grid container may include a reflector positioned on or in a hole corresponding to the aperture-axis such that the reflector is orthogonal to the aperture-axis, and the apparatus further includes: a camera; and a beam splitter, wherein a light source is configured to emit a light beam through the beam splitter, wherein the beam splitter is configured to direct a first portion of the light beam toward the camera, wherein the beam splitter is configured to direct a second portion of the light beam toward the reflector, wherein the reflector is configured to reflect the second portion of the light beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the light beam toward the camera.

[0323] Example 3: An apparatus that can implement any of the foregoing examples, wherein the beam splitter can be a cubic prism including a first beam splitting coating and a second beam splitting coating, wherein the first beam splitting coating can be positioned along the inner diagonal plane of the cubic prism, and wherein the second beam splitting coating can be positioned along the outer surface of the cubic prism.

[0324] Example 4: An apparatus that can implement any of the foregoing examples, the apparatus further comprising: a collimating lens positioned between the light source and the beam splitter.

[0325] Example 5: An apparatus that can implement any of the foregoing examples, wherein the camera is configured to capture an image that depicts a first illumination spot caused by a first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portions of the light beam.

[0326] Example 6: An apparatus can be implemented in any of the preceding examples, wherein the non-concentricity of the first and second illumination spots indicates that the axis-axis of the robot gripper is not angularly or translationally aligned with the hole-axis of the microscope grid container, and wherein the concentricity of the first and second illumination spots indicates that the axis-axis of the robot gripper is angularly or translationally aligned with the hole-axis of the microscope grid container.

[0327] Example 7: An apparatus that can implement any of the foregoing examples, wherein the processor is configured to apply object detection technology to an image to determine whether the first illumination spot and the second illumination spot overlap.

[0328] Example 8: An apparatus that can implement any of the foregoing examples, wherein in response to determining that the first illumination spot and the second illumination spot do not overlap, the processor may be configured to: identify a first centroid of the first illumination spot based on the average position of pixels belonging to the first illumination spot; identify a second centroid of the second illumination spot based on the average position of pixels belonging to the second illumination spot; calculate, via one or more kinematic formulas, an angle or translational position that would reduce the distance between the first centroid and the second centroid from which they are separated; and move the robot gripper to that angle or translational position.

[0329] Example 9: An apparatus that can implement any of the foregoing examples, wherein, in response to determining that the first illumination spot and the second illumination spot overlap, the processor can be configured to: define the first illumination spot and the second illumination spot with a minimum area profile; identify a first centroid of the first illumination spot and a second centroid of the second illumination spot via one or more geometric formulas based on the minimum area profile; in response to a distance between the first centroid and the second centroid being greater than a threshold, calculate an angle or translational position by which the robot gripper would predict reducing that distance via one or more kinematic formulas; and move the robot gripper to that angle or translational position.

[0330] Example 10: An apparatus that can implement any of the foregoing examples, wherein, in response to determining that the first illumination spot and the second illumination spot overlap, the processor is configured to: define the first illumination spot and the second illumination spot with a minimum area profile; identify a first centroid of the first illumination spot and a second centroid of the second illumination spot via one or more geometric formulas based on the minimum area profile; in response to a distance less than a threshold separating the first centroid and the second centroid, cause a robot gripper to sweep across a series of angular or translational positions within a threshold proximity of its current angular or translational position; calculate the corresponding area of ​​the minimum area profile for the series of angular or translational positions; and move the robot gripper to any of the series of angular or translational positions corresponding to the minimum area of ​​the minimum area profile.

[0331] In various implementation schemes, any one or more combinations of Examples 1 to 10 may be implemented.

[0332] Example 11: A computer-implemented method may include: accessing a charged particle microscope having a vacuum chamber via a device operatively coupled to a processor, wherein the vacuum chamber may include a robotic gripper and a microscope grid container; and via the device, teaching the robotic gripper to align its axis-axis with the aperture-axis of the microscope grid container angularly or translationally, based on a light source coupled to the robotic gripper.

[0333] Example 12: A computer-implemented method of any of the foregoing examples may be implemented, the method further comprising: through the device, causing the light source to emit a light beam through a beam splitter, wherein the beam splitter is configured to direct a first portion of the light beam toward a camera, wherein the beam splitter is configured to direct a second portion of the light beam toward a reflector positioned on or in an aperture corresponding to an aperture-axis, wherein the reflector is configured to reflect the second portion of the light beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the light beam toward the camera.

[0334] Example 13: A computer-implemented method of any of the foregoing examples can be implemented, wherein the beam splitter can be a cubic prism including a first beam splitting coating and a second beam splitting coating, wherein the first beam splitting coating can be positioned along the inner diagonal plane of the cubic prism, and wherein the second beam splitting coating can be positioned along the outer surface of the cubic prism.

[0335] Example 14: A computer-implemented method of any of the foregoing examples can be implemented, wherein the camera is configured to capture an image that depicts a first illumination spot caused by a first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portions of the light beam.

[0336] Example 15: A computer-implemented method that can implement any of the foregoing examples, the method further comprising: determining, via the device and by applying object detection technology to the image, whether a first illumination spot and a second illumination spot overlap.

[0337] Example 16: A computer-implemented method that can implement any of the foregoing examples, the method further comprising, in response to determining that a first illumination spot and a second illumination spot do not overlap: identifying a first centroid of the first illumination spot based on the average position of pixels belonging to the first illumination spot using the device; identifying a second centroid of the second illumination spot based on the average position of pixels belonging to the second illumination spot using the device; calculating, using the device and via one or more kinematic formulas, an angle or translational position that would reduce the distance between the first and second centroids by a predicted distance using the device; and moving the robot gripper to the angle or translational position using the device.

[0338] Example 17: A computer-implemented method that can implement any of the foregoing examples, the method further comprising, in response to determining that a first illumination spot and a second illumination spot overlap: defining the first illumination spot and the second illumination spot with a minimum area profile using the device; identifying a first centroid of the first illumination spot and a second centroid of the second illumination spot using the device and via one or more geometric formulas based on the minimum area profile; in response to a distance between the first centroid and the second centroid being greater than a threshold, calculating an angle or translational position of the robot gripper that would reduce the distance using the device and via one or more kinematic formulas; and moving the robot gripper to the angle or translational position using the device.

[0339] Example 18: A computer-implemented method of any of the foregoing examples, the method further comprising, in response to determining that a first illumination spot and a second illumination spot overlap: defining the first illumination spot and the second illumination spot with a minimum area profile via the device; identifying a first centroid of the first illumination spot and a second centroid of the second illumination spot via the device and via one or more geometric formulas based on the minimum area profile; in response to a distance less than a threshold separating the first centroid and the second centroid, sweeping a series of angular or translational positions within a threshold proximity of its current angular or translational position via the device; calculating a corresponding area of ​​the minimum area profile for the series of angular or translational positions via the device; and moving the robot gripper via the device to any one of the series of angular or translational positions corresponding to the minimum area of ​​the minimum area profile.

[0340] In various implementation schemes, any one or more combinations of Examples 11 to 18 can be implemented.

[0341] Example 19: A computer program product for facilitating the teaching of a robotic gripper for a charged particle microscope via beam splitting may include a non-transitory computer-readable storage device having program instructions embodied therein. In various aspects, the program instructions are executable by a processor to cause the processor to: access a charged particle microscope having a vacuum chamber, wherein the vacuum chamber may include a robotic gripper and a microscope grid container; cause a light beam emitted by a light source attached to the robotic gripper to pass through a beam splitter, wherein the beam splitter is configured to direct a first portion of the beam toward a camera, wherein the beam splitter is configured to direct a second portion of the beam toward a reflector positioned on or within an aperture in the microscope grid container, wherein the reflector is configured to reflect the second portion of the beam back to the beam splitter, and wherein the beam splitter is configured to... Directing at least some of the second portion of the light beam toward the camera; accessing an image captured by the camera, wherein the image depicts a first illumination spot caused by the first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portion of the light beam; based on the distance between the corresponding centroids of the first and second illumination spots, identifying one or more angular or translational movements of the robotic gripper that would align the axis-to-axis of the robotic gripper angularly or translatively with the aperture of the microscope mesh container; and causing the robotic gripper to perform one or more angular or translational movements.

[0342] Example 20: A computer program product that can implement any of the foregoing examples, wherein how the processor analyzes the image to identify the one or more angular or translational movements can be based on whether the first illumination spot and the second illumination spot overlap.

[0343] In various implementation schemes, any one or more combinations of Examples 19 to 20 can be implemented.

[0344] In various implementation schemes, any one or more combinations of Examples 1 to 20 can be implemented.

Claims

1. An apparatus comprising: A charged particle microscope with a vacuum chamber, wherein the vacuum chamber includes a robotic gripper and a microscope grid container; and A processor configured to teach the robotic gripper, based on a light source coupled to the robotic gripper, to align its axis-to-axis with the aperture-to-axis of the microscope mesh container angularly or translationally.

2. The apparatus of claim 1, wherein the microscope grid container includes a reflector positioned on or within a hole corresponding to the aperture-axis such that the reflector is orthogonal to the aperture-axis, and the apparatus further comprises: camera; and A beam splitter, wherein the light source is configured to emit a light beam through the beam splitter, wherein the beam splitter is configured to direct a first portion of the light beam toward the camera, wherein the beam splitter is configured to direct a second portion of the light beam toward the reflector, wherein the reflector is configured to reflect the second portion of the light beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the light beam toward the camera.

3. The apparatus of claim 2, wherein the beam splitter is a cubic prism comprising a first beam splitting coating and a second beam splitting coating, wherein the first beam splitting coating is positioned along the inner diagonal plane of the cubic prism, and wherein the second beam splitting coating is positioned along the outer surface of the cubic prism.

4. The apparatus according to claim 2, further comprising: A collimating lens is positioned between the light source and the beam splitter.

5. The apparatus of claim 2, wherein the camera is configured to capture an image, wherein the image depicts a first illumination spot caused by the first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portion of the light beam.

6. The apparatus of claim 5, wherein the non-concentricity of the first illumination spot and the second illumination spot indicates that the axis-axis of the robot gripper is not angularly or translationally aligned with the hole-axis of the microscope mesh container, and wherein the concentricity of the first illumination spot and the second illumination spot indicates that the axis-axis of the robot gripper is angularly or translationally aligned with the hole-axis of the microscope mesh container.

7. The apparatus of claim 5, wherein the processor is configured to apply object detection technology to the image to determine whether the first illumination spot and the second illumination spot overlap.

8. The apparatus according to claim 7, wherein, In response to determining that the first illumination spot and the second illumination spot do not overlap, the processor is configured to: Based on the average position of the pixels belonging to the first illumination spot, the first centroid of the first illumination spot is identified; Based on the average position of the pixels belonging to the second illumination spot, the second centroid of the second illumination spot is identified; The predicted angle or translational position of the robot gripper, calculated using one or more kinematic formulas, would reduce the distance between the first and second centers of mass. Move the robot gripper to the angle or translation position.

9. The apparatus according to claim 7, wherein, In response to determining that the first illumination spot and the second illumination spot overlap, the processor is configured to: The first illumination spot and the second illumination spot are defined by the minimum area contour; Based on the minimum area profile, the first centroid of the first illumination spot and the second centroid of the second illumination spot are identified via one or more geometric formulas; In response to a distance greater than a threshold separating the first and second centroids, the robot gripper calculates, via one or more kinematic formulas, the predicted angle or translational position that would reduce that distance; and Move the robot gripper to the angle or translation position.

10. The apparatus according to claim 7, wherein, In response to determining that the first illumination spot and the second illumination spot overlap, the processor is configured to: The first illumination spot and the second illumination spot are defined by the minimum area contour; Based on the minimum area profile, the first centroid of the first illumination spot and the second centroid of the second illumination spot are identified via one or more geometric formulas; In response to the distance between the first centroid and the second centroid being less than a threshold, the robot gripper sweeps a series of angular or translational positions within a threshold proximity of its current angular or translational position. Calculate the corresponding area of ​​the minimum area contour for the series of angles or translation positions; and Move the robot gripper to any of the series of angular or translational positions that corresponds to the minimized area of ​​the minimum area profile.

11. A computer-implemented method, comprising: Access to a charged particle microscope with a vacuum chamber via a device operatively coupled to the processor, wherein the vacuum chamber includes a robotic gripper and a microscope grid container; as well as Using the device, based on a light source coupled to the robotic gripper, the robotic gripper is taught to align its axis-to-axis with the hole-to-axis of the microscope mesh container angularly or translationally.

12. The computer-implemented method according to claim 11, further comprising: The device allows the light source to emit a beam that passes through a beam splitter, wherein the beam splitter is configured to direct a first portion of the beam toward a camera, wherein the beam splitter is configured to direct a second portion of the beam toward a reflector positioned on or in an aperture corresponding to the aperture-axis, wherein the reflector is configured to reflect the second portion of the beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the beam toward the camera.

13. The computer-implemented method of claim 12, wherein the beam splitter is a cubic prism comprising a first beam-splitting coating and a second beam-splitting coating, wherein the first beam-splitting coating is positioned along an inner diagonal plane of the cubic prism, and wherein the second beam-splitting coating is positioned along an outer surface of the cubic prism.

14. The computer-implemented method of claim 12, wherein the camera is configured to capture an image, wherein the image depicts a first illumination spot caused by the first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portion of the light beam.

15. The computer-implemented method according to claim 14, further comprising: The device and object detection technology applied to the image determine whether the first illumination spot and the second illumination spot overlap.

16. The computer-implemented method of claim 15, further comprising, in response to determining that the first illumination spot and the second illumination spot do not overlap: Using the device, the first centroid of the first illumination spot is identified based on the average position of the pixels belonging to the first illumination spot; Using the device, the second centroid of the second illumination spot is identified based on the average position of the pixels belonging to the second illumination spot; The device, along with one or more kinematic formulas, calculates the predicted angle or translational position of the robot gripper that would reduce the distance between the first and second centers of mass separation; and The device allows the robot gripper to be moved to the stated angle or translation position.

17. The computer-implemented method of claim 15, further comprising, in response to determining that the first illumination spot and the second illumination spot overlap: The device is used to define the first illumination spot and the second illumination spot using a minimum area contour. The first centroid of the first illumination spot and the second centroid of the second illumination spot are identified by the device and via one or more geometric formulas based on the minimum area profile. In response to the distance between the first centroid and the second centroid being greater than a threshold, the robot gripper is calculated, via the device and through one or more kinematic formulas, to predict that the distance will be reduced by an angle or translational position. The device allows the robot gripper to be moved to the stated angle or translation position.

18. The computer-implemented method of claim 15, further comprising, in response to determining that the first illumination spot and the second illumination spot overlap: The device is used to define the first illumination spot and the second illumination spot using a minimum area contour. The first centroid of the first illumination spot and the second centroid of the second illumination spot are identified by the device and via one or more geometric formulas based on the minimum area profile. In response to the distance between the first centroid and the second centroid being less than a threshold, the device causes the robot gripper to sweep across a series of angular or translational positions within a threshold proximity of its current angular or translational position. Using the device, the corresponding area of ​​the minimum area contour is calculated for the series of angles or translational positions; and The device allows the robot gripper to be moved to any of the series of angular or translational positions that correspond to the minimum area of ​​the minimum area profile.

19. A computer program product for facilitating the teaching of a charged particle microscope robotic gripper via beam splitting, the computer program product comprising a non-transitory computer-readable storage memory having program instructions embodied therein, the program instructions being executable by a processor to cause the processor to: Access a charged particle microscope with a vacuum chamber, wherein the vacuum chamber includes a robotic gripper and a microscope grid container; A light beam emitted by a light source attached to the robot gripper is passed through a beam splitter, wherein the beam splitter is configured to direct a first portion of the beam toward a camera, wherein the beam splitter is configured to direct a second portion of the beam toward a reflector positioned on or in an aperture of the microscope grid container, wherein the reflector is configured to reflect the second portion of the beam back to the beam splitter, and wherein the beam splitter is configured to direct at least some of the second portion of the beam toward the camera; Access an image captured by the camera, wherein the image depicts a first illumination spot caused by the first portion of the light beam, and wherein the image depicts a second illumination spot caused by at least some of the second portion of the light beam; Based on the distance between the corresponding centroids of the first and second illumination spots, identify one or more angular or translational movements of the robotic gripper that would align the axis-to-axis of the robotic gripper angularly or translationally with the aperture of the microscope mesh container; and The robot gripper performs one or more angular or translational movements.

20. The computer program product of claim 19, wherein how the processor analyzes the image to identify the one or more angles or translational movements is based on whether the first illumination spot and the second illumination spot overlap.

Images