Sample positioning based on deep learning

Abstract

Deep learning based sample positioning. Scientific instrument support systems, and related methods, computing devices, and computer readable media are disclosed herein. For example, in some embodiments, a method for determining sample positions and associated stage coordinates by a microscope includes at least: acquiring, using a navigation camera, an image of a plurality of samples loaded on a clamp, the image having a low resolution at a field of view that includes the clamp and all of the plurality of samples; analyzing, using a trained model, the image to identify the plurality of samples; based on the analysis, associating each sample with a position on the clamp; based on the position of each sample on the clamp, associating individual stage coordinate information with each of the plurality of samples loaded on the clamp; and translating a stage holding the clamp to a first stage coordinate based on the associated stage coordinate information of a first sample of the plurality of samples.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to charged particle microscope lenses, and in particular to charged particle microscope objectives that produce negligible or zero magnetic field at the sample plane in the direction of the optical axis. BACKGROUND

[0002] Microscopes are used in many industries for quality control, defect detection, process analysis, etc., so that companies can understand their processes and characterize them accordingly. For example, the semiconductor industry uses various microscope tools, such as charged particle microscopes (e.g., scanning electron microscopes (SEMs), focused ion beam (FIB) microscopes, dual-beam combining both SEM and FIB, transmission electron microscopes (TEMs), and scanning TEMs (STEMs)), to analyze their processes and resulting devices. The use of such microscopes on samples taken from larger process batches, such as wafer batches, wafers, etc., has historically been a highly manual task of identifying, tracking, and transporting such samples. However, with current advances in robotics and control algorithms, it is desirable to automate sample identification, tracking, and handling, which would allow skilled technicians to perform more value-added tasks than loading samples and tracking them through various microscope processes. BRIEF DESCRIPTION OF DRAWINGS

[0003] The various embodiments will be described in detail in the following description with reference to the drawings. For the purpose of description, the same reference numerals will be used throughout the description to designate the same elements. The various embodiments are illustrated by way of example in the drawings and will be described in detail in the following description.

[0004] Figure 1A is an illustrative block diagram of a charged particle microscope (CPM) support module 1000 for performing sample-related operations according to various embodiments disclosed herein.

[0005] Figure 1B is an illustrative block diagram of a charged particle microscope (CPM) support module 1001 for performing sample-related operations according to various embodiments disclosed herein.

[0006] Figure 2 is a flowchart of a method 2000 for performing support operations according to various embodiments.

[0007] Figure 3 is a flowchart of a method 3000 of performing support operations according to various embodiments.

[0008] Figure 4 is a block diagram of a computing device 4000 that can perform some or all of the scientific instrument support methods disclosed herein according to various embodiments.

[0009] Figure 5is a block diagram of an example scientific instrument support system 5000 in which some or all of the scientific instrument support methods disclosed herein can be performed in accordance with various embodiments.

[0010] Figure 6 is an example gripper 6000 for loading multiple samples into a CPM in accordance with embodiments of the present disclosure.

[0011] Figure 7 is an example CPM 100 in accordance with embodiments of the present disclosure. DETAILED DESCRIPTION

[0012] Scientific instrument support systems, and related methods, computing devices, and computer-readable media are disclosed herein. For example, in some embodiments, a method for determining sample locations and associated stage coordinates by a microscope comprises at least: acquiring, using a navigation camera, an image of a plurality of samples loaded on a gripper, the image having a low resolution at a field of view that includes the gripper and all of the plurality of samples; analyzing the image using a trained model to identify the plurality of samples; based on the analyzing, associating each sample with a location on the gripper; based on the location of each sample on the gripper, associating individual stage coordinate information with each of the plurality of samples loaded on the gripper; and translating a stage holding the gripper to a first stage coordinate based on the associated stage coordinate information of a first sample of the plurality of samples.

[0013] In another embodiment, the disclosed technology comprises at least generating a job file for a plurality of samples, the job file comprising at least a sample ID and a process associated with each sample of the plurality of samples; loading the plurality of samples onto a gripper, the gripper comprising a plurality of gripper locations, wherein each gripper location can accept one sample, and wherein each gripper location is at a known location on the gripper and has an associated identifier, and wherein the gripper comprises a marker that orients the gripper locations; updating the job file to associate each sample of the plurality of samples with a gripper location; loading the gripper into a charged particle microscope (CPM), the gripper mounted on a stage of the CPM, and the stage moved so that the gripper is in a position that coincides with a charged particle beam and a navigation camera (navcam); acquiring an image of the gripper using the navcam; providing the image to a trained model for analysis; receiving, from the trained model, stage coordinate information for each sample loaded on the gripper; updating the job file to associate the stage coordinate information for each sample; and initiating the job file to process each sample.

[0014] The scientific instrument support embodiments disclosed herein can enable improved performance relative to conventional approaches. For example, the extreme manual requirements of tracking and identifying previous samples in a microscope prior to initiating a specified process have been removed by the disclosed technology. For example, the disclosed technology includes using a trained model to identify samples in images taken in a microscope, identifying each sample based on its location on a holder, and associating stage coordinates with each identified sample based on its holder location. The microscope then uses the stage coordinates to move the stage to process each sample. Such automation allows a skilled technician to perform more valuable tasks associated with the data and / or control a set of microscopes rather than one or two. As such, the embodiments disclosed herein provide improvements to scientific instrument technology (e.g., improvements to computer technology aspects that support such scientific instruments, among other improvements).

[0015] Various of the embodiments disclosed herein can improve conventional approaches to enable higher throughput, more accurate data associated with samples, and technical advantages of increased efficiency by automating the steps of identifying samples loaded on a holder, associating samples with specific stage coordinates stored in a relevant job file that will be used to automatically process each sample on the holder. Once sample processing begins, the system can automatically navigate to each sample location due to the stored stage coordinates, without requiring a user to perform the navigation. Such technical advantages are not achievable by routine and conventional approaches, and all users of systems that include such embodiments can benefit from these advantages (e.g., by helping users perform technical tasks, such as automated sample identification and stage coordinate association through guided human-machine interaction processes). As such, the technical features of the embodiments disclosed herein are decidedly unconventional in the field of microscopes, and particularly charged particle microscopes, as are the combinations of features of the embodiments disclosed herein. As discussed elsewhere herein, various aspects of the embodiments disclosed herein can improve the functionality of the computer itself; for example, by providing one or more trained models that allow the computer to perform the disclosed technology that conventional algorithms are not capable of performing. The computing and user interface features disclosed herein not only involve the collection and comparison of information, but also apply new analytical and technical techniques to change the operation of a charged particle microscope. Thus, the present disclosure introduces functionality that neither conventional computing devices nor humans are capable of performing.

[0016] Accordingly, the embodiments of the present disclosure can serve any of a variety of technical purposes, such as controlling a particular technical system or process; determining how to control a machine as a function of a measurement; and / or digital audio, image, or video enhancement or analysis. In particular, the present disclosure provides technical solutions to technical problems, including but not limited to automated sample identification and associating stage locations with each identified sample.

[0017] In the following detailed description, references are made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration various embodiments that can be practiced. The same reference numerals are used throughout the drawings and written description to refer to like elements. It is to be understood that other embodiments can be utilized, and structural or logical changes can be made without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be taken in a limiting sense.

[0018] Various operations can be described as multiple discrete actions or operations, in turn, in a manner that is most helpful in understanding the subject matter described herein. However, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations can not be performed in the order of presentation. Operations described can be performed in a different order than the described embodiment. Various additional operations can be performed and / or described operations can be omitted in additional embodiments.

[0019] For purposes of the present disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For purposes of the present disclosure, the phrases "A, B, and / or C" and "A, B, or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C). Although some elements can be represented in the singular (e.g., "processing device"), any appropriate number of instances of the element can be represented by the element, and vice versa. For example, a set of operations described as being performed by a processing device can be implemented by different processing devices performing different ones of the operations.

[0020] Each of the descriptions using the phrases "one embodiment," "various embodiments," and "some embodiments" can refer to one or more of the same or different embodiments. Furthermore, the terms "comprising," "including," "containing," etc. shall be construed as synonymous and open ended whenever they are used in a description of embodiments of the present disclosure. The phrase "between X and Y" when used in a description of a range of dimensions, refers to the range including X and Y. As used herein, "device" can refer to any individual apparatus, collection of apparatuses, portion of an apparatus, or collection of portions of apparatuses. The drawings are not necessarily to scale.

[0021] Figure 1A and 1B Illustrative block diagrams of charged particle microscope (CPM) support modules 1000 and 1001 for performing sample-related operations in accordance with various embodiments disclosed herein. The CPM support modules 1000, 1001 can be implemented by circuitry (e.g., including electrical and / or optical components) of a computing device as programmed. The logic of the CPM support modules 1000 can be contained in a single computing device or can be distributed across multiple computing devices in communication with each other as appropriate. Reference is made herein to Figure 4The computing device 4000 discussed above in connection with FIG. 4 can be an example of a computing device that can implement alone or in combination with one or more other computing devices the CPM support module 1000, 1001, and the CPM support system 5000 discussed herein with reference to Figure 5 The CPM support system 5000 discussed above in connection with FIG. 5 is an example of a system of interconnected computing devices in which the CPM support module 1000, 1001 can be implemented across one or more of the computing devices.

[0022] The CPM support module 1000 can include first logic 1002, second logic 1004, third logic 1006, and (optional) fourth logic 1008. The CPM support module 1001 can include fifth logic 1010, sixth logic 1012, and seventh logic 1014. As used herein, the term "logic" can include a device that performs a set of operations associated with the logic. For example, any of the logic elements included in the support module 1000, 1001 can be implemented by one or more computing devices programmed with instructions to cause one or more processing devices in the computing device to perform the associated set of operations. In particular embodiments, a logic element can include one or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processing devices in one or more computing devices, cause the one or more computing devices to perform the associated set of operations. As used herein, the term "module" can refer to a collection of one or more logic elements that together perform the functions associated with the module. Different ones of the logic elements in a module can take the same form or can take different forms. For example, some of the logic in a module can be implemented by a general-purpose processing device programmed with instructions, while other logic in the module can be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module can be associated with different sets of instructions executed by one or more processing devices. A module can not include all of the logic elements depicted in the associated diagram; for example, the module can include a subset of the logic elements depicted in the associated diagram when that module is to perform a subset of the operations discussed herein with reference to that module.

[0023] The first logic 1002 can include data handling logic that can receive one or more job files. A job file can include a sample identification number (sample ID) and an associated process to be performed by the CPM on the sample. Additionally, a job file can include information that associates a sample with a location on which the sample is loaded on a gripper, which can be referred to as a sample location or gripper location, as will be discussed in greater detail below. In general, the first logic 1002 can handle data received from external and internal sources and store the data with the correct sample, where the data file includes information about each sample loaded on a particular gripper.

[0024] The second logic 1004 can include CPM control routines for carrying out various tasks of CPM execution. For example, the second logic 1004 can include control routines for loading a chuck into the CPM and moving the chuck to a position under the SEM column where it can also be seen by at least one other camera, such as a navigation camera. The second control logic 1004 can then cause images of the chuck to be acquired using the navigation camera and / or SEM. The navigation camera images can be referred to herein as navigation camera images. In addition to movement of the stage, the second logic 1004 can include control routines for performing imaging using the navigation camera or SEM using various parameters. Example imaging parameters for the SEM can include at least electron beam energy, electron beam dwell time, electron beam raster rate, image magnification, resolution, and field of view, to name a few. Regardless of the image processing, once determined, the control logic 1004 can access the job file to move the stage to the particular stage coordinates associated with each sample so that the imaging process associated with each sample can be performed. Those skilled in the art will be aware of and understand the various control features of a CPM and will further recognize that the above discussion is not an exhaustive list of the various control aspects.

[0025] The third logic 1006 can be responsible for providing and receiving information, data, and / or images to / from users and / or processing logic located outside of the CPM. For example, the third logic 1006 can cause at least the navigation camera images to be provided to processing logic coupled to the CPM via a network, such as a local area network, a wide area network, or the World Wide Web. Alternatively or additionally, the third logic 1006 can provide data, information, and / or images to the fourth logic 1008. Further, the third logic 1006 can also update the job file with process data, e.g., images, for each sample as the process is performed or after it is performed. Generally, the third logic 1006 is input / output logic to control the flow of information within and to and from the support module 1000.

[0026] The optional fourth logic 1008 can include one or more deep learning based models trained to identify the samples and their locations in the navigation camera images and the stage positions associated with each sample. The stage positions inform the CPM where to move the stage to align each sample with the SEM column of the CPM. As will be discussed in more detail below, the fixture can include markings formed thereon that allow the fourth logic 1008 to determine the orientation of the fixture so that the pre-identified individual sample locations are located and allow the fourth logic 1008 to associate the stage positions with each sample. It should be noted that the fixture can have a set number of sample locations, each of which has been pre-associated with a sample ID, such that the job file associates each sample ID with a sample location on the fixture. In this way, the fourth logic 1008 determines the location of each sample location relative to the fixture markings and provides the associated stage positions for each sample to the first logic 1004 for addition to the job file.

[0027] Figure 1B The fifth logic 1010 can be input / output control logic configured to receive and provide data as needed. For example, the fifth logic 101 can receive the navigation camera images from the third logic 1006.

[0028] The sixth logic 1012 can include one or more logic features that work in conjunction to analyze the navigation camera images and provide stage coordinates for each identified sample. For example, the various processing logic of the sixth logic 1012 can be broken down into separate logic components 1014, 1016, 1018, and 1020. However, it should be understood that this division is merely for illustration and that more, different, or fewer divisions can be made. The sixth logic 1012 can include ML or AI trained models for segmenting the received images, such as the navigation camera images, so that sample locations and samples can be identified in the images. In some embodiments, the sixth logic 1012 can be broken into multiple logic modules, as noted, with each module performing a separate function, the end goal being to provide stage coordinates for each sample identified in the navigation camera images. For example, the seventh logic 1014 can simply segment the images and provide the segmented images to separate logic for additional analysis, such as to the eighth logic 1016 that identifies the samples in the images. Upon identifying the samples, the ninth logic 1018 can determine the location of each identified sample on the fixture. And finally, the tenth logic 1020 can determine the stage position for each sample based on the location of each sample on the fixture.

[0029] Generally, the AI and ML models can be trained to identify the samples via image segmentation, e.g., determine the location of each sample relative to pre-known sample locations on the fixture, and then associate stage coordinates with each sample based on the location of each sample. Such stage coordinate information can then be provided to the job file.

[0030] In some examples, both support modules 1000 and 1001 can be included in the CPM and provide the disclosed logical functionality in response to input controls. However, in other embodiments, support module 1000 can be included in the CPM while support module 1001 can be located separately, such as at a remote server accessed over any type of network. In yet other embodiments, both support modules 1000 and 1001 can be located remotely from the CPM and accessed over a network.

[0031] Figure 2 A flowchart for a method 2000 for performing support operations in accordance with various embodiments is shown. Although the operations of method 2000 can be described with reference to the particular embodiments disclosed herein (e.g., the CPM support modules 1000, 1001 discussed with reference to Figure 1A and 1B the computing device 4000 discussed with reference to Figure 4 and / or the scientific instrument support system 5000 discussed with reference to Figure 5 , the method 2000 can be used in any suitable setting to perform any suitable support operations. The operations are illustrated once and in a particular order, but the operations can be reordered and / or repeated as desired and as appropriate (e.g., different operations can be performed in parallel as appropriate). Figure 2

[0032] At 2002, a first operation can be performed. In some examples, the first operation 2002 can be performed manually by a user, but in other examples, the first operation 2002 can be performed automatically, e.g., by a robotic instrument. The first operation can include loading samples onto a gripper. For example, a plurality of samples can be loaded into gripper positions as shown as positions 1-9 in Figure 6 .

[0033] At 2004, a second operation can be performed. Like the first operation 2002, the second operation 2004 can be performed manually by a user, but in other examples, the second operation 2004 can be performed automatically, e.g., by a robotic instrument. The second operation can include loading the gripper into the CPM. The gripper, on which the plurality of samples will be loaded, will be loaded, e.g., through a load lock, and placed on or interfaced with a movable stage of the CPM. The loading of the gripper will additionally include movement of the stage to place the gripper in the imaging position of both the charged particle beam and the low resolution, large field of view camera of the CPM. The low resolution, large field of view camera can be a navigation camera as discussed above, and will be arranged within the CPM to image the gripper when it is in coincidence with the charged particle beam.

[0034] ​At point 2006, a third operation can be performed. For example, the second logic 1004 of support module 1000 can perform the operation 2006. The third operation 2006 may involve acquiring an image of the fixture containing the sample using a navigation camera. Such an image may only have grayscale values ​​and will provide a large field-of-view image of the entire fixture, which contains all the samples loaded thereon. Such an image may be referred to herein as a navigation camera image.

[0035] At point 2008, a fourth operation can be performed. For example, the third logic 1006 of support module 1000 can perform the operation 2008. The fourth operation 2008 may include providing navigation camera images to a training model for analysis. The training model may be a deep learning model, a machine learning model, a neural network model, etc., trained to recognize samples mounted on a fixture, associate sample positions with each sample, and provide stage coordinates based thereon. The stage coordinates will be used by the CPM to align the associated samples with the charged particle beam for imaging and / or processing. As will be discussed below, the fixture may include markings that allow the training model to orient the fixture using sample positions and their labels. For example, each fixture position may be numbered, for example, so that the training model can associate each sample with a fixture position.

[0036] At point 2010, a fifth support operation can be performed. For example, the first logic 1002 can update the job file based on the stage coordinates of each sample mounted on the fixture, using the analysis performed by the trained model.

[0037] At point 2012, a sixth support operation can be performed. For example, second logic 1004 can perform a process associated with each sample as indicated in the job file, and may include converting the stage to the sample's stage coordinates before performing the associated process with each sample.

[0038] The scientific instrument support methods disclosed herein may include interactions with human users (e.g., through the references herein). Figure 5 The user's local computing device 5020 is discussed. These interactions may include providing information to the user (e.g., about such...). Figure 5 Information on the operation of scientific instruments such as the Scientific Instrument 5010, information about the sample to be analyzed or other tests or measurements performed by the scientific instrument, information retrieved from local or remote databases, or other information, or providing users with options to input commands (e.g., for controlling such...). Figure 5scientific instrument 5010, etc.) or other information. In some embodiments, these interactions can be performed through a graphical user interface (GUI) that is included in the output provided to the user and / or that prompts the user for input (e.g., through a display device discussed above with reference to Figure 4 one or more of the other I / O devices 4012 discussed above, such as a keyboard, mouse, trackpad, or touchscreen). The scientific instrument support system disclosed herein can include any suitable GUI for interacting with a user. Figure 4

[0039] Figure 3 A flowchart of a method 3000 for performing support operations in accordance with various embodiments. Although the operations of the method 3000 can be described with reference to the particular embodiments disclosed herein (e.g., the CPM support module 1000, 1001 discussed above with reference to Figure 1A and 1B the computing device 4000 discussed above with reference to Figure 4 and / or the scientific instrument support system 5000 discussed above with reference to Figure 5 the computing device 4000 discussed above with reference to Figure 3 the scientific instrument support system 5000 discussed above with reference to

[0040] At 3002, a first operation can be performed. In some examples, the first operation 3002 can include acquiring an image of a gripper, where the image is acquired using a navigation camera, as discussed above. The gripper can have one or more samples loaded in specified locations on the gripper, and can have been loaded onto a stage of a CPM. For example, a plurality of samples can be loaded into gripper locations 1-9 as shown in Figure 6

[0041] ​​At 3004, a second operation can be performed. The second operation can include analyzing the image to identify the samples loaded onto the gripper, which can be performed by the fourth logic 1008 or the sixth logic 1012 and / or the seventh logic 1014. The analysis can be performed by a ML or AI based trained model that has been trained to at least segment the image and identify the samples loaded on the gripper. In some embodiments, the model is trained to associate each identified sample with a known position on the gripper, which can be determined based on markers on the gripper. These markers can provide orientation information to the model. Additionally, based on the designated gripper positions for accepting samples, this is part of the training. The gripper, on which multiple samples will be loaded, will be loaded, for example, through a load lock, and placed on or interfaced with a movable stage of the CPM. The loading of the gripper will additionally include movement of the stage to place the gripper in the imaging position of both the charged particle beam and the low resolution, large field of view camera of the CPM. The low resolution, large field of view camera can be a navigation camera as discussed above, and will be arranged within the CPM to image the gripper when it is in coincidence with the charged particle beam.

[0042] At 3006, a third operation can be performed. For example, the fourth logic 1004, the sixth logic 1012, and / or the ninth logic 1018 of the support module 1000, 1001 can perform the operations of 3006. The third operation 3006 can include the trained model of the functional logic analyzing the navigation camera image and determining the position of each sample relative to the pre-known gripper positions, associating each sample with a gripper position.

[0043] At 3008, a fourth operation can be performed. The fourth operation 3008 can include associating stage coordinate information with each sample based on the position of each sample on the gripper. The trained model of the fourth logic 1008, the sixth logic 1012, or the tenth logic 1020 can perform the analysis, respectively. The stage coordinates will be used by the CPM to align the associated samples with the charged particle beam for imaging and / or processing. As will be discussed below, the gripper can include markers that allow the trained model to use the sample positions, as well as their labels, to orient the gripper. For example, each gripper position can be numbered, for example, so that the trained model can associate each sample with a gripper position.

[0044] At 3010, a fifth support operation can be performed. For example, the fifth logic 1010 can provide the stage coordinate information to the second logic 1004, which will update the job file using the stage coordinates of each sample loaded on the gripper based on the analysis performed by the trained model.

[0045] At 3012, a sixth support operation can be performed. For example, the first logic 1002 can perform a process associated with each sample as indicated in the job file, and can include converting the stage to the stage coordinates of the sample before performing the associated process with each sample.

[0046] The scientific instrument support methods disclosed herein may include interactions with human users (e.g., through the references herein). Figure 5 The user's local computing device 5020 is discussed. These interactions may include providing information to the user (e.g., about such...). Figure 5 Information on the operation of scientific instruments such as the Scientific Instrument 5010, information about the sample to be analyzed or other tests or measurements performed by the scientific instrument, information retrieved from local or remote databases, or other information, or providing users with options to input commands (e.g., for controlling such...). Figure 5 The operation of scientific instruments such as the scientific instrument 5010 or for controlling and analyzing data generated by the scientific instrument, querying (e.g., querying a local or remote database), or other information. In some embodiments, these interactions can be performed through a graphical user interface (GUI) that includes providing output to the user and / or prompting the user for input (e.g., via methods included in this reference). Figure 4 Other I / O devices discussed 4012 include one or more input devices such as a keyboard, mouse, touchpad, or touchscreen, and a display device (e.g., referred to herein). Figure 4 The visual display on the display device (4010) discussed herein. The scientific instrument support system disclosed herein may include any suitable GUI for user interaction.

[0047] As described above, the scientific instrument support modules 1000 and 1001 can be implemented by one or more computing devices. Figure 4 This is a block diagram of a computing device 4000 capable of performing some or all of the scientific instrument support methods disclosed herein, according to various embodiments. In some embodiments, scientific instrument support modules 1000, 1001 may be implemented by a single computing device 4000 or multiple computing devices 4000. Additionally, as discussed below, the computing device 4000 (or multiple computing devices 4000) implementing the scientific instrument support module 1000 may be... Figure 5 A portion of one or more of the scientific instrument 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040.

[0048] Figure 4The computing device 4000 is illustrated as having a number of components, any one or more of which can be omitted or duplicated, as appropriate for a given application and setting. In some embodiments, some or all of the components included in the computing device 4000 can be attached to one or more motherboards and enclosed within a housing (e.g., comprising plastic, metal, and / or other materials). In some embodiments, some of these components can be fabricated onto a single system on a chip (SoC) (e.g., the SoC can include one or more processing devices 4002 and one or more storage devices 4004). Additionally, in various embodiments, the computing device 4000 can not include one or more of the components illustrated in FIG. D, but can include interface circuitry (not shown) for coupling to one or more components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI) interface, a controller area network (CAN) interface, a serial peripheral interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, the computing device 4000 can not include the display device 4010, but can include display device interface circuitry (e.g., a connector and driver circuitry) to which a display device 4010 can be coupled.

[0049] The computing device 4000 can include a processing device 4002 (e.g., one or more processing devices). As used herein, the term “processing device” can refer to any device or portion of a device that processes electronic data from registers and / or memory to transform that electronic data into other electronic data that can be stored in registers and / or memory. The processing device 4002 can include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (specialized processors that perform cryptographic algorithms within hardware), server processors, or any other suitable processing devices.

[0050] The computing device 4000 can include a storage device 4004 (e.g., one or more storage devices). The storage device 4004 can include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive-bridging RAM (CBRAM) devices), hard-disk drive-based memory devices, solid-state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 4004 can include memory that shares a die with the processing device 4002. In such embodiments, the memory can function as cache memory and can include, for example, embedded dynamic random access memory (eDRAM) or spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, the storage device 4004 can include a non-transitory computer-readable medium having instructions thereon, which, when executed by one or more processing devices (e.g., the processing device 4002), cause the computing device 4000 to perform any appropriate method or portion of a method disclosed herein.

[0051] The computing device 4000 can include an interface device 4006 (such as one or more interface devices 4006). The interface device(s) 4006 can include one or more communication chips, connectors, and / or other hardware and software to manage communications between the computing device 4000 and other computing devices. For example, the interface device(s) 4006 can include circuitry for managing the transfer of data to and from the computing device 4000 through wireless communication. The term “wireless” and its derivatives can be used to describe circuits, devices, systems, methods, techniques, communications channels, and / or the like that can communicate data through the use of modulated electromagnetic radiation through a non-solid medium. The term does not imply that the associated devices do not contain any wires, although in some embodiments they might not. The circuitry included in the interface device(s) 4006 for managing wireless communication can implement any of a number of wireless standards or protocols, including but not limited to IEEE standards including Wi-Fi (the IEEE 802.11 family of standards), IEEE 802.16 standards (e.g., IEEE 802.16-2005 Amendment), the Long-Term Evolution (LTE) project

[0052] In some embodiments, the interface device 4006 can include circuitry for managing wired communication, such as electrical, optical, or any other suitable communication protocol. For example, the interface device 4006 can include circuitry that supports communication according to Ethernet technology. In some embodiments, the interface device 4006 can support both wireless and wired communication, and / or can support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry of the interface device 4006 can be dedicated to short-range wireless communication, such as Wi-Fi or Bluetooth, and a second set of circuitry of the interface device 4006 can be dedicated to long-range wireless communication, such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, a first set of circuitry of the interface device 4006 can be dedicated to wireless communication, and a second set of circuitry of the interface device 4006 can be dedicated to wired communication.

[0053] The computing device 4000 can include a battery / power supply circuitry 4008. The battery / power supply circuitry 4008 can include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 4000 to a power source separate from the computing device 4000 (e.g., an AC line power source).

[0054] The computing device 4000 can include a display device 4010 (e.g., a plurality of display devices). The display device 4010 can include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0055] The computing device 4000 can include other input / output (I / O) devices 4012. The other I / O devices 4012 can include, for example, one or more audio output devices (e.g., speakers, headphones, earbuds, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), positioning devices (e.g., a GPS device that communicates with satellite-based systems to receive a location of the computing device 4000 as known in the art), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), image capture devices such as cameras, keyboards, cursor control devices such as mice, styluses, trackballs, or touchpads, bar code readers, quick response (QR) code readers, or radio frequency identification (RFID) readers.

[0056] The computing device 4000 may have any suitable form factor for its application and setup, such as a handheld or mobile computing device (e.g., a cellular phone, smartphone, mobile internet device, tablet computer, laptop computer, netbook computer, ultrabook computer, personal digital assistant (PDA), ultra-mobile personal computer, etc.), a desktop computing device, or a server computing device or other network computing component.

[0057] One or more computing devices that implement any of the scientific instrument support modules or methods disclosed herein may be part of a scientific instrument support system. Figure 5 This is a block diagram of an example scientific instrument support system 5000, according to various embodiments, in which some or all of the scientific instrument support methods disclosed herein can be performed. The scientific instrument support modules and methods disclosed herein (e.g., Figure 1A and 1B Scientific instrument support modules 1000 and 1001, Figure 2 Method 2000 and Figure 3 Method 3000 can be implemented by one or more of the scientific instrument 5010, user local computing device 5020, service local computing device 5030 or remote computing device 5040 of the scientific instrument support system 5000.

[0058] Any of the scientific instrument 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may contain the information described herein. Figure 4 Any of the embodiments of the computing device 4000 discussed herein, and any of the scientific instrument 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may employ the methods described herein. Figure 4 The form of any suitable embodiment of the computing device 4000 discussed in the embodiments.

[0059] Scientific instrument 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may each include a processing unit 5002, a storage unit 5004, and an interface unit 5006. The processing unit 5002 may take any suitable form, including those described herein. Figure 4 The processing device 5002 described herein may take any of the forms, and the processing device 5002 included in different devices of the scientific instrument 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may take the same or different forms. The storage device 5004 may take any suitable form, containing the information described herein. Figure 4The storage device 5004 discussed herein may take any of the forms, and the storage device 5004 included in different devices in the scientific instrument 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may take the same or different forms. The interface device 5006 may take any suitable form, containing the information discussed herein. Figure 4 The interface device 5006 discussed may take any of the forms, and the interface device 5006 included in different devices such as scientific instrument 5010, user local computing device 5020, service local computing device 5030 or remote computing device 5040 may take the same or different forms.

[0060] Scientific instrument 5010, user local computing device 5020, service local computing device 5030, and remote computing device 5040 can communicate with other components of scientific instrument support system 5000 via communication path 5008. Communication path 5008 can communicatively couple with interface devices 5006 of different components within scientific instrument support system 5000, as shown, and can be a wired or wireless communication path (e.g., according to the provisions of this document regarding...). Figure 4 (The interface device 4006 of the computing device 4000 is any of the communication technologies discussed). The particular scientific instrument support system 5000 depicted in Figure E includes communication paths between each pair of scientific instruments 5010, user local computing device 5020, service local computing device 5030, and remote computing device 5040. However, this “fully connected” implementation is merely illustrative, and in various embodiments, various communication paths in communication path 5008 may not exist. For example, in some embodiments, the service local computing device 5030 may not have a direct communication path 5008 between its interface device 5006 and the interface device 5006 of the scientific instrument 5010, but may communicate with the scientific instrument 5010 via a communication path 5008 between the service local computing device 5030 and the user local computing device 5020 and a communication path 5008 between the user local computing device 5020 and the scientific instrument 5010.

[0061] Scientific Instruments 5010 may include any suitable scientific instrument, such as Figure 7 The CPM discussed in the article.

[0062] The user local computing device 5020 can be a computing device that is local to a user of the scientific instrument 5010 (e.g., according to any of the embodiments of the computing device 4000 discussed herein). In some embodiments, the user local computing device 5020 can also be local to the scientific instrument 5010, although this is not necessarily the case; for example, a user local computing device 5020 located in a user's home or office can be remote from the scientific instrument 5010, but in communication with the scientific instrument, such that a user can use the user local computing device 5020 to control and / or access data from the scientific instrument 5010. In some embodiments, the user local computing device 5020 can be a laptop computer, a smartphone, or a tablet computing device. In some embodiments, the user local computing device 5020 can be a portable computing device. In some embodiments, the user local computing device 5020 can control initiation of the technology disclosed herein and also allow a user to control various tasks of the CPM.

[0063] The service local computing device 5030 can be a computing device (e.g., according to any of the embodiments of the computing device 4000 discussed herein) that is local to an entity that services the scientific instrument 5010. For example, the service local computing device 5030 can be local to a manufacturer of the scientific instrument 5010 or a third-party service company. In some embodiments, the service local computing device 5030 can communicate with the scientific instrument 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., via the direct communication path 5008 or via multiple “indirect” communication paths 5008, as discussed above) to receive data regarding the operation of the scientific instrument 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., self-test results of the scientific instrument 5010, calibration coefficients used by the scientific instrument 5010, measurements of sensors associated with the scientific instrument 5010, etc.). In some embodiments, the service local computing device 5030 can communicate with the scientific instrument 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., via the direct communication path 5008 or via multiple “indirect” communication paths 5008, as discussed above) to transmit data to the scientific instrument 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., to update programmed instructions, such as firmware, in the scientific instrument 5010, to initiate performance of a test or calibration sequence in the scientific instrument 5010, to update programmed instructions, such as software, in the user local computing device 5020 or the remote computing device 5040, etc.). A user of the scientific instrument 5010 can utilize the scientific instrument 5010 or the user local computing device 5020 to communicate with the service local computing device 5030 to report problems with the scientific instrument 5010 or the user local computing device 5020, to request access from a technician to improve operation of the scientific instrument 5010, to order consumable or replaceable components associated with the scientific instrument 5010, or for other purposes. The service local computing device 5030 can allow for remote control of a CPM, such as initiation of the techniques disclosed herein, and / or updating of training of a training model.

[0064] The remote computing device 5040 can be a computing device (e.g., according to any of the embodiments of the computing device 4000 discussed herein) that is remote from the scientific instrument 5010 and / or the user local computing device 5020. In some embodiments, the remote computing device 5040 can be included in a data center or other large server environment. In some embodiments, the remote computing device 5040 can include network attached storage (e.g., as part of the storage device 5004). The remote computing device 5040 can store data generated by the scientific instrument 5010, perform analysis on data generated by the scientific instrument 5010 (e.g., according to programmed instructions), facilitate communication between the user local computing device 5020 and the scientific instrument 5010, and / or facilitate communication between the service local computing device 5030 and the scientific instrument 5010. The remote computing device can include instances of trained models and can perform image analysis and stage coordinate association, as disclosed herein.

[0065] In some embodiments, one or more of the elements of the scientific instrument support system 5000 described in FIG. 5A can not be present. Additionally, in some embodiments, one or more elements not described in FIG. 5A can be present. Figure 5 In some embodiments, one or more of the elements of the scientific instrument support system 5000 described in FIG. 5A can not be present. Additionally, in some embodiments, one or more elements not described in FIG. 5A can be present. Figure 5The scientific instrument support system 5000 can include multiple elements of various elements of the scientific instrument support system 5000. For example, the scientific instrument support system 5000 can include multiple user local computing devices 5020 (e.g., different user local computing devices 5020 associated with different users or in different locations). In another example, the scientific instrument support system 5000 can include multiple scientific instruments 5010 all in communication with a service local computing device 5030 and / or a remote computing device 5040; in such embodiments, the service local computing device 5030 can monitor these multiple scientific instruments 5010, and the service local computing device 5030 can cause updates or other information to be “broadcast” to the multiple scientific instruments 5010 simultaneously. Different ones of the scientific instruments 5010 in the scientific instrument support system 5000 can be located close to one another (e.g., in the same room) or far from one another (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, the scientific instruments 5010 can be connected to an Internet of Things (IoT) stack that allows the scientific instruments 5010 to be commanded and controlled through web-based applications, virtual or augmented reality applications, mobile applications, and / or desktop applications. Any of these applications can be accessible by a user operating a user local computing device 5020 that communicates with the scientific instruments 5010 through an intervening remote computing device 5040. In some embodiments, a manufacturer can sell a scientific instrument 5010 with one or more associated user local computing devices 5020 as part of a local scientific instrument computing unit 5012.

[0066] In some embodiments, different ones of the scientific instruments 5010 included in the scientific instrument support system 5000 can be different types of scientific instruments 5010; for example, one scientific instrument 5010 can be a SEM, while another scientific instrument 5010 can be a dual-beam FIB-SEM. In some such embodiments, the remote computing device 5040 and / or the user local computing devices 5020 can combine data from different types of scientific instruments 5010 included in the scientific instrument support system 5000.

[0067] Figure 6 An example fixture 6000 for loading multiple samples into a CPM according to embodiments of the present disclosure. The fixture 6000 includes a body 6002 with pre-described sample locations 6004. Figure 6Embodiments of the present disclosure show nine (9) preset sample locations, but the number of such locations is not limiting and any number of locations are within the scope of the present disclosure. Each sample location 6004 can be configured to accept a sample that is either manually or robotically loaded onto the gripper 6000 by a robotic device such as a robotic arm. Additionally, each sample location 6004 can include some means of holding the sample in place, such as a mechanical clip, a groove, or a non-slip type surface.

[0068] The gripper 6000 can also include markers 6006 and 6008 that are used to orient the gripper 6000, at least in image analysis. While the gripper 6000 shows only two markers, additional markers at different locations on the gripper are possible and are contemplated herein. In some embodiments, the markers 6006 are different from the markers 6008 to help determine the gripper orientation. In other embodiments, a third marker can be added to help identify an orientation where all three markers are the same. In either embodiment, the markers can be large enough to be easily captured in a navigation camera image, such as a low resolution, wide field of view, grayscale image, etc.

[0069] Figure 7 An example CPM 100 according to embodiments of the present disclosure. For example, the CPM 100 can be used to implement the techniques disclosed herein to identify samples loaded on a gripper and associate each identified sample with stage coordinates. Additional features as discussed above, such as job file creation and updating, will also be implemented on the CPM 100 to perform the techniques disclosed herein. Generally, the CPM 100 will have a gripper with samples loaded therein and images of the gripper are captured and analyzed to identify the samples and associate each sample with stage coordinates and sample locations on the gripper.

[0070] The CPM 100 can include at least an emitter 102, an optical column 104, a camera 116, a stage 114, and a controller 118. The emitter 102 and optical column 104 can be housed in a column housing 106. In some embodiments, the column housing 106 and the components contained therein form a scanning electron microscope (SEM). While the CPM 100 can be discussed herein as a SEM, other CPM types are also contemplated, such as a focused ion beam (FIB) system, a (scanning) transmission electron microscope, and a dual beam system containing both a SEM and a FIB. Generally, the type of microscope is non-limiting and even an optical microscope can fall within the scope of an instrument implementing the disclosed techniques. The emitter can be configured to emit a charged particle beam 108 that can be conditioned by the optics 104 and directed toward the stage 114.

[0071] The stage 114 and camera 116 can be housed in a vacuum compatible enclosure 110 that includes a load lock (not shown) for loading and unloading samples and / or clamps as discussed herein. The stage 114 can be a multi-axis stage capable of movement in 3, 4, or 5 dimensions such that a clamp / sample loaded on the stage can be translated and rotated to align the sample with a charged particle beam (CPB) such as the CPB 108. The camera 116 can be a simple detector that is capable of acquiring images of the stage and clamp loaded thereon when the stage is under the CPB 108. Typically, the camera 116 can be positioned to capture images that coincide with the images captured by the SEM column, but at much lower resolution. The camera 116 can acquire low resolution grayscale images at a wide field of view to ensure that at least the entire clamp is captured in the image. In some embodiments, the camera 116 can be a navigation camera as discussed above, which is well known in the art of charged particle microscopy. However, it should be understood that the type of camera used for the camera 116 is non-limiting.

[0072] The controller 118 can be coupled to the memory 120 and additionally coupled to control the various components of the CPM 100 such as the emitter 102, optics 104, stage 114, and camera 116. Data collected by the CPM 100 can also be provided to the controller 118 for analysis, control feedback, or for providing to additional computing components connected to the CPM 100 over one or more networks. Typically, the controller 118 can be configured as a hardware Figure 5 and contain logic Figure 1A and 1B for implementing the techniques disclosed herein. The memory 120 can contain operational instructions for implementing the techniques disclosed herein and additionally can contain one or more trained models for image analysis and provide stage coordinates for each sample identified by the image analysis.

Claims

1. A method for determining the position of a sample and associated stage coordinates using a microscope, the method comprising: A navigation camera is used to acquire images of multiple samples mounted on a fixture, the images having low resolution in the field of view including the fixture and all samples of the multiple samples; The trained model is used to analyze the images to identify the multiple samples; Based on the analysis, each sample is associated with a position on the fixture; Based on the position of each sample on the fixture, the individual stage coordinate information is associated with each of the plurality of samples loaded on the fixture; as well as Based on the associated stage coordinate information of the first sample among the plurality of samples, the stage of the fixture is kept translated to the first stage coordinate.

2. The method according to claim 1, further comprising: Associate the process with the sample under the coordinate information of the first stage; as well as The process is executed when the stage is translated to the coordinates of the first stage.

3. The method of claim 1, further comprising: The job file is received at the microscope or by a server coupled to the microscope. The job file includes a sample ID, a sample number on the fixture, and process information. as well as Access the job file to associate the process with the sample.

4. The method of claim 1, wherein analyzing the image using a trained model to identify the plurality of samples comprises: The trained model is used to segment the image; as well as Each of the plurality of samples is identified based on the segmentation.

5. The method of claim 4, further comprising: The orientation of the clamp is determined based on markings on the clamp, wherein the markings are arranged to indicate the orientation of the clamp; Associate each sample with a position on the fixture; as well as Each sample is associated with the stage position and associated stage coordinates.

6. The method of claim 5, wherein the clamp includes a plurality of the marks arranged across the clamp.

7. The method of claim 1, wherein the navigation camera is arranged to observe a position that coincides with the observation position of the optics of the microscope.

8. The method of claim 7, wherein the microscope is a charged particle microscope and the optics include charged particle optics.

9. The method of claim 1, further comprising: After the process is performed using the stage at the first stage coordinate, the stage holding the fixture is translated to the second stage coordinate.

10. The method of claim 9, wherein the process includes acquiring a scanned electronic image.

11. A method comprising: Multiple samples are loaded onto a fixture, the fixture including multiple fixture positions, each fixture position capable of accepting one sample, and each fixture position being at a known location on the fixture and having an associated identifier, and wherein the fixture includes markings for the orientation of the fixture positions; Update the working file to associate each of the plurality of samples with a fixture location, wherein each sample has an associated sample ID; The fixture is loaded into a charged particle microscope (CPM), the fixture is mounted on the stage of the CPM, and the stage is moved so that the fixture is in a position that coincides with the charged particle beam and the navigation camera (navcam). The navigation camera is used to acquire an image of the clamp; The images are provided to the training model for analysis; Receive stage coordinate information for each sample mounted on the fixture from the training model; Update the working file to associate the stage coordinate information for each sample; as well as The workfile is initiated to process each sample, wherein the workfile associates the process with each of the plurality of samples.

12. The method of claim 11, wherein the training model is a machine learning model trained to identify samples and associate each sample with the stage position.

13. The method of claim 11, wherein each sample ID identifies the sample and its associated batch number, wafer number, and die number.

14. The method of claim 11, further comprising: The training model receives a sample identifier for each of the plurality of fixture positions.

15. The method of claim 11, further comprising: A working file is generated for multiple samples, the working file including at least the sample ID and the process associated with each of the multiple samples.

16. The method of claim 11, wherein the stage is a multi-axis stage.

17. The method of claim 11, wherein the clamp includes a plurality of marks for determining the orientation of the clamp on the stage.

18. The method of claim 11, wherein the CPM comprises at least a scanning electron microscope (SEM) column for performing the process on each of the plurality of samples.

19. The method of claim 18, wherein the process is capable of imaging the sample using a SEM column at the energy level, field of view, and magnification indicated in the working document.

20. The method of claim 11, wherein each sample is a semiconductor sample.

Images