Sheet endpoint localization via graph weighted neural networks

Abstract

This disclosure provides systems or techniques for facilitating slice endpoint localization via a graph weighted neural network. In various embodiments, a system can access an image captured by a scientific instrument, where the image depicts a section of a sheet. In various aspects, the system can generate a classification tag for the section via execution of a graph weighted neural network, where the classification tag can indicate which of a plurality of defined classes the section belongs to. In various cases, the system can command the scientific instrument to incrementally mill the section of the sheet in response to the classification tag indicating that the section does not belong to a target class of the plurality of defined classes.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 585,891, entitled “WEIGHTED GRAPH REINFORCED NEURAL NETWORK ENDPOINTING,” filed on September 27, 2023. The entire contents of the foregoing application are hereby incorporated herein by reference. Background Art

[0003] Various scientific instruments can perform milling and imaging of thin slices, which, when used in conjunction with artificial neural networks, can facilitate end-pointing of such slices. Summary of the invention

[0004] The following presents a summary of the invention to provide a basic understanding of one or more embodiments. The summary of the invention is not intended to identify key or important elements, or to delineate any scope of a particular embodiment or any scope of the claims. Its sole purpose is to present concepts in a simplified form as a prelude to a more detailed description presented later. In one or more embodiments described herein, an apparatus, system, computer-implemented method, device, or computer program product is described that facilitates thin film endpoint positioning via a graph weighted neural network.

[0005] According to one or more embodiments, a system is provided. The system may include a non-transitory computer readable memory that may store a computer executable component. The system may also include a processor that may be operably coupled to the non-transitory computer readable memory and may execute the computer executable component stored in the non-transitory computer readable memory. In various embodiments, the computer executable component may include an access component that may access an image captured by a scientific instrument, wherein the image may depict a section of a thin slice. In various aspects, the computer executable component may include a model component that may generate a classification label for the section via execution of a graph weighted neural network, wherein the classification label may indicate which of a plurality of defined classes the section belongs to. In various cases, the computer executable component may include a milling component that may command the scientific instrument to incrementally mill the section of the thin slice in response to the classification label indicating that the section does not belong to a target class in the plurality of defined classes.

[0006] According to one or more embodiments, a computer-implemented method is provided. In various embodiments, the computer-implemented method may include accessing, by a device operatively coupled to a processor, an image captured by a scientific instrument, wherein the image may depict a cross-section of a thin slice. In various aspects, the computer-implemented method may include generating, by the device and via execution of a graph weighted neural network, a classification label for the cross-section, wherein the classification label may indicate to which of a plurality of defined classes the cross-section belongs. In various cases, the computer-implemented method may include commanding, by the device and in response to the classification label indicating that the cross-section does not belong to a target class of the plurality of defined classes, the scientific instrument to incrementally mill the cross-section of the thin slice.

[0007] According to one or more embodiments, a computer program product for facilitating thin slice endpoint positioning via a graph weighted neural network is provided. In various embodiments, the computer program product may include a non-transitory computer readable memory having program instructions contained therein. In various aspects, the program instructions may be executed by a processor to cause the processor to access an image captured by a dual-beam charged particle microscope, wherein the image may depict a cross-section of a thin slice. In various cases, the program instructions may be executed to cause the processor to generate a classification label for the cross-section via execution of the graph weighted neural network, wherein the classification label may indicate which of a plurality of structures the cross-section contains. In various cases, the program instructions may be executed to cause the processor to command the dual-beam charged particle microscope to incrementally mill the cross-section of the thin slice in response to the classification label indicating that the cross-section does not contain a target structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Various embodiments will be easily understood by the following detailed description in conjunction with the accompanying drawings. For ease of description, the same reference numerals indicate the same structural elements. The embodiments are illustrated in the figures by way of example and not by way of limitation. These figures are not necessarily drawn to scale.

[0009] Figure 1 An example non-limiting block diagram of a scientific instrument module is illustrated in accordance with various embodiments described herein.

[0010] Figure 2 An example non-limiting flow chart illustrating a computer-implemented method according to various embodiments described herein.

[0011] Figure 3 Illustrated is a block diagram of an example non-limiting system that facilitates sheet endpoint localization via a graph weighted neural network in accordance with one or more embodiments described herein.

[0012] Figure 4Illustrated is a block diagram of an example non-limiting system including a graph weighted neural network and classification labels to facilitate sheet endpoint localization via a graph weighted neural network according to one or more embodiments described herein.

[0013] Figures 5 to 9 An example non-limiting block diagram showing how a graph weighted neural network may be used to generate classification labels according to one or more embodiments described herein is illustrated.

[0014] Fig.10 A block diagram illustrating an example non-limiting system including target classes and milling instructions to facilitate sheet endpoint positioning via a graph weighted neural network according to one or more embodiments described herein.

[0015] Figure 11 to Figure 12 A flow chart illustrating an example non-limiting computer-implemented method for facilitating sheet endpoint localization via a graph weighted neural network according to one or more embodiments described herein.

[0016] Fig.13 Illustrated is a block diagram of an example non-limiting system that facilitates sheet endpoint localization via a graph weighted neural network according to one or more embodiments described herein including a training component and a training data set.

[0017] Figure 14 to Figure 15 An example non-limiting block diagram showing how a neural network may be trained is illustrated in accordance with one or more embodiments described herein.

[0018] Figures 16 to 20 Example non-limiting experimental results according to one or more embodiments described herein are illustrated.

[0019] Fig.21 An example non-limiting block diagram of a graphical user interface that may be used to perform some or all of the methods or techniques disclosed herein, according to various embodiments described herein.

[0020] Fig. 22 An example non-limiting block diagram of a computing device that can perform some or all of the methods or techniques disclosed herein is illustrated in accordance with various embodiments described herein.

[0021] Fig.23 An example non-limiting block diagram of a scientific instrument support system in which some or all of the methods or techniques disclosed herein may be performed in accordance with various embodiments described herein is illustrated.

[0022] Fig.24 A block diagram illustrating an example non-limiting operating environment in which one or more embodiments described herein may be facilitated.

[0023] Fig.25 An example networking environment is illustrated that is operable to perform various implementations described herein.

[0024] Fig.26 An example dual-beam microscope that may be implemented in accordance with various embodiments described herein is illustrated. DETAILED DESCRIPTION

[0025] The following detailed description is illustrative only and is not intended to limit the embodiments and / or the application or uses of the embodiments. In addition, there is no intention to be bound by any express or implied information presented in the previous background or summary or detailed description sections.

[0026] One or more embodiments are now described with reference to the accompanying drawings, wherein the same reference numerals are used throughout to refer to the same elements. In the following description, for the purpose of explanation, many specific details are set forth to provide a more thorough understanding of one or more embodiments. However, it is apparent that in various circumstances, one or more embodiments may be practiced without these specific details.

[0027] Various operations may be described as multiple discrete actions or operations in a manner that is most helpful for understanding the subject matter disclosed herein. However, the described order should not be interpreted as implying that these operations must rely on the order. In particular, these operations may be performed in an order different from the order of presentation. The described operations may be performed in an order different from the described embodiments. Various additional operations may be performed, or the described operations may be omitted in additional embodiments.

[0028] Although some elements may be referred to in singular form (e.g., "processing device"), any appropriate element may be represented by multiple instances of the element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented as different ones of the operations being performed by different processing devices. As used herein, the phrase "based on" should be understood to mean "based at least in part on," unless otherwise specified.

[0029] A scientific instrument (e.g., a mass spectrometer, a charged particle microscope) can be any suitable computerized device that can capture or generate electronic measurements (e.g., can capture or generate spectral images or component spectra) in a scientific, laboratory, research, or clinical operating context. To facilitate the capture or generation of such electronic measurements, a scientific instrument can utilize a complex arrangement of actuatable components (e.g., ion sources, ion lenses, heaters, coolers, fluid valves, fluid pumps, circuit switches, sample stages, apertures), sensors (e.g., ion detectors, voltmeters, thermistors, potentiometers, manometers), or consumables (e.g., carrier fluids, calibrants, filters).

[0030] Various scientific instruments (e.g., scanning or transmission electron microscopes, electron energy loss microscopes) can utilize their constituent actuatable components to capture images of samples (e.g., electron tomography images, X-ray tomography images, confocal microscopy images). In some cases, those images may be two-dimensional arrays of pixels. In other cases, those images may instead be three-dimensional arrays of voxels. In addition to being able to capture images of samples, various scientific instruments (e.g., dual-beam charged particle microscopes) can perform milling (e.g., material removal or etching) on ​​such samples.

[0031] In various cases, the ability of various scientific instruments to perform both imaging and milling on samples can be utilized to facilitate thin slice endpoint positioning. In particular, a thin slice can be a thin rectangular layer or slice cut from a larger manufacturing sample (e.g., cut from a printed circuit board or from a semiconductor substrate). In various cases, a thin slice may contain one or more structures or substructures of interest. For example, a thin slice may contain a transistor that is constructed or otherwise built into or on a larger manufacturing sample (e.g., via any suitable lithography or deposition technology), and the transistor itself may be considered to be composed of a gate, a source, a drain, and a fin. In various aspects, it may be desirable to determine whether various structures or substructures (e.g., gates, source, drain, or fins) in those structures or substructures are correctly or otherwise constructed or constructed as expected. In order to facilitate such determination, it may be desirable to first physically separate a given one of those structures or substructures or substructures, and then analyze the given structure or substructure (e.g., via a transmission electron microscope). In various aspects, the separation of the given structure or substructure can be facilitated by a process called thin slice endpoint positioning.

[0032] In various cases, lamella endpoint location may involve: a scientific instrument capable of both imaging and milling a lamella; and an artificial neural network that is trained or otherwise configured to classify structures or substructures of a particular lamella (e.g., two different lamellae containing different or otherwise dissimilar types of structures or substructures may require two differently trained or configured artificial neural networks). More specifically, the scientific instrument may incrementally mill a first side (e.g., the front side) of a lamella (e.g., removing material from the first side) (e.g., via an ion beam of a dual-beam microscope), thereby forming a cut surface on the first side of the lamella. The scientific instrument may then capture an image of the cut surface after milling (e.g., via an electron beam of a dual-beam microscope). Next, the image may be fed to the artificial neural network, and the artificial neural network may generate a classification label that indicates which particular structure or substructure of the lamella is depicted in the image. If the classification label indicates that the depicted structure or substructure is different from any of the structures or substructures that are desired to be physically separated, the various operations mentioned above can be repeated (e.g., the scientific instrument can again incrementally mill and image the cut surface, and the artificial neural network can generate a new classification label indicating whether the desired structure or substructure has been reached). Conversely, if the classification label indicates that the depicted structure or substructure is the desired structure or substructure, it can be concluded that the slice endpoint positioning with respect to the first side of the sheet is completed or ended. Then, the slice endpoint positioning can be performed with respect to the second side of the sheet (e.g., the back side). Once the slice endpoint positioning with respect to the second side is completed or ended, it can be considered that the desired structure or substructure is now physically separated. In other words, it can be considered that all other structures or substructures of the sheet have been iteratively removed from the sheet via milling.

[0033] Thus, slice endpoint positioning can be considered an iterative milling and imaging process that gradually thins the slice until the slice contains only any one particular structure or substructure that is desired to be separated (e.g., before endpoint positioning, the slice may have a thickness measuring approximately 50 nanometers (nm) to 130 nm; after endpoint positioning, the slice may have a thickness measuring approximately 25 nm or less, depending on the thickness of the structure or substructure that is desired to be separated).

[0034] Unfortunately, the prior art for facilitating the location of the endpoints of a thin slice has various disadvantages. In particular, any artificial neural network for performing the location of the endpoints of a thin slice can be first trained in a supervised manner using training images and respectively corresponding real data classification labels. When implementing some prior art, the artificial neural network can be trained on a large number of image-annotation pairs (e.g., tens of thousands of training images, each of which has a corresponding real data classification label). Although such prior art can enable the artificial neural network to achieve a high level of classification accuracy, such prior art unfortunately involves sacrificing a lot of thin slices. In fact, tens or tens of image-annotation pairs can be obtained by iteratively imaging and milling a single thin slice. Therefore, in order to train the artificial neural network to achieve a satisfactory or sufficient level of classification accuracy, some prior art needs to sacrifice or destroy hundreds of thin slices. This may be considered to be excessively wasteful (e.g., thin slices are layers or slices of real-world samples, which have non-zero costs associated with their manufacture or construction; therefore, it can be considered that destroying thin slices in order to create endpoint location training data is expensive). When implementing other prior art, the artificial neural network can be trained on much fewer image-annotation pairs instead. While such other prior art techniques may involve sacrificing or destroying a much smaller number of flakes (e.g., and therefore may be considered less wasteful), such other prior art artificial neural networks often fail to achieve a satisfactory or sufficient level of classification accuracy (e.g., with too little training, the artificial neural network fails to properly learn how to recognize flake structures or substructures). Such low accuracy may be undesirable.

[0035] Therefore, systems or techniques that can ameliorate one or more of these technical problems may be desirable.

[0036] Various embodiments described herein can solve one or more of these technical problems. One or more embodiments described herein may include a system, a computer-implemented method, an apparatus, or a computer program product that can facilitate the location of a thin slice endpoint via a graph weighted neural network. In particular, the various embodiments described herein can facilitate or otherwise perform endpoint location for a given thin slice by utilizing an artificial neural network, the classification labels of which are modified (e.g., weighted) by multiplication via a neighborhood graph corresponding to a given thin slice. More specifically, as mentioned above, it can be considered that a given thin slice contains various structures or substructures of interest, and it can be considered that endpoint location physically separates specific structures or substructures in those structures or substructures, so that it can be determined whether those separated structures or substructures are properly manufactured or constructed. In other words, the structure or substructure of interest can be manufactured or constructed according to a specific engineering design or schematic diagram, and it can therefore be expected that the structure or substructure of interest is physically located or placed in a thin slice along a milling direction or milling axis in a certain order defined by the specific engineering design or schematic diagram. In other words, it can be expected that certain structures or substructures of interest are encountered during endpoint positioning before other structures or substructures of interest are encountered (e.g., it can be expected that the gate of a transistor is encountered during endpoint positioning before the source and drain of the transistor is encountered; it can be expected that the source and drain of a transistor are encountered during endpoint positioning before the fin of the transistor is encountered). In various aspects, the neighborhood graph can be a directed graph (e.g., including nodes and directed edges) that represents or otherwise captures such positional or orientational ordering of structures or substructures within a given slice. In particular, the nodes of the neighborhood graph can respectively represent structures or substructures within a given slice, each node can be coupled to all other nodes via directed edges, and the weights of those directed edges can respectively indicate which of those structures or substructures are or are not neighbors of each other along the milling direction or milling axis (e.g., edges coupling neighbors can be given greater weights, and edges coupling non-neighbors can be given lower weights). In various cases, each time the artificial neural network generates a classification label for an image of a sheet, the structure-wise probability in the classification label can be multiplied by the corresponding weight in the weights in the neighborhood graph. This can result in a modified classification label. As described herein, the modified classification label (rather than the original classification label directly output by the artificial neural network) can be used to trigger the termination of the sheet endpoint positioning. As experimentally verified by the inventors of the various embodiments described herein, it can be believed that weighting the artificial neural network via the neighborhood graph in this manner improves the classification accuracy of the artificial neural network. Therefore, the artificial neural network can be trained on a small data set without sacrificing or destroying many sheets without suffering from poor classification accuracy.This contrasts with prior art techniques which force a choice between high accuracy at the expense of many flakes and fewer flakes at the expense of reduced classification accuracy.

[0037] The various embodiments described herein may be considered to be computerized tools (e.g., any suitable combination of computer executable hardware or computer executable software) that can facilitate slice endpoint positioning via a graph weighted neural network. In various aspects, such a computerized tool may include an access component, a model component, or a milling component.

[0038] In various embodiments, there may be a scientific instrument. In various aspects, the scientific instrument may be any suitable computerized device that can perform or facilitate both milling and imaging of the thin slice. As a non-limiting example, the scientific instrument may be a dual-beam microscope that can facilitate milling via an ion beam and can facilitate imaging via an electron beam. In various cases, the thin slice may be a thin rectangular slice or layer cut from any suitable larger sample. In various cases, the thin slice may contain or otherwise include any suitable structure of interest. As a non-limiting example, if the thin slice is cut from a semiconductor circuit sample, the thin slice may contain a transistor constructed onto the semiconductor circuit sample via photolithography, and the transistor may be composed of one or more gates, one or more source drains, and one or more fins. In this example, one or more gates, one or more source drains, and one or more fins may be considered to be structures of interest.

[0039] In various aspects, the scientific instrument can electronically capture an image of the slice. In various cases, the image can be a two-dimensional array of pixels that visually depicts a slice facet. In various cases, the facet can be a surface of the slice (e.g., front surface; back surface) from which material that was previously part of the slice was removed by the scientific instrument via a previous milling operation. Thus, the facet can be considered to be a cross-section of the slice that has been exposed via milling. Note that performing more or additional milling can be considered to move the facet deeper into the slice. The slice can contain different structures of interest at different depths, and thus different ones of those structures can be visible in or on the facet, depending on the depth of the facet (e.g., depending on how much total or cumulative milling has been performed on the slice).

[0040] In any case, it may be desirable to perform endpoint positioning on a slice based on an image of a cut surface. That is, it may be desirable to determine whether a slice has been sufficiently milled so that a desired structure is contained or otherwise present in or on a cut surface. As described herein, a computerized tool may facilitate such endpoint positioning.

[0041] In various embodiments, the access component of the computerized tool can access the image of the section electronically. For example, the access component can receive, retrieve or otherwise obtain the image from any suitable centralized or decentralized data structure (e.g., a graph data structure, a relational data structure, a hybrid data structure). As a non-limiting example, the access component can receive, retrieve or obtain the image from the scientific instrument itself. In any case, the access component can be considered to be a conduit through which other components of the computerized tool can interact with the image electronically (e.g., read, write, edit, copy, manipulate the image).

[0042] In various embodiments, the model component of the computerized tool can electronically store, maintain, control or otherwise access the deep learning neural network. In various aspects, the deep learning neural network can present any suitable deep learning internal architecture. For example, the deep learning neural network may include any suitable number of layers of any suitable type (e.g., input layer, one or more hidden layers, output layer, any of which can be a convolutional layer, a dense layer, a nonlinear layer, a long short-term memory (LSTM) layer, a pooling layer, a batch normalization layer, or a padding layer). As another example, the deep learning neural network may include any suitable number of neurons in various layers (e.g., different layers may have the same or different numbers of neurons). As another example, the deep learning neural network may include any suitable activation function (e.g., softmax, sigmoid, hyperbolic tangent, rectified linear unit) in various neurons (e.g., different neurons may have the same or different activation functions). As another example, the deep learning neural network may include any suitable inter-neuron connection or inter-layer connection (e.g., forward connection, skip connection, recurrent connection).

[0043] Regardless of its particular internal architecture, the deep learning neural network can be configured as an image classifier corresponding to a particular structure contained within the sheet. In particular, the deep learning neural network can be configured to classify which structures, if any, of any structures contained within the sheet are depicted or illustrated in an input image. Thus, the model component can electronically execute the deep learning neural network on an image of a slice of the sheet, and such execution can cause the deep learning neural network to produce a raw classification label. More specifically, the model component can feed the image of the slice to an input layer of the deep learning neural network, the image of the slice can complete a forward pass through one or more hidden layers of the deep learning neural network, and an output layer of the deep learning neural network can operate or calculate the raw classification label based on activations provided by the one or more hidden layers.

[0044] In various aspects, the original classification label can be a vector whose numerical elements indicate the probability that a corresponding structure among the structures contained within the slice is depicted in the image of the section. As a non-limiting example, assume that for any suitable positive integer n, it is known that the slice contains n different structures. In this case, the original classification label may include a total of n probabilities corresponding to those n structures, respectively. In various cases, the first probability value of the original classification label may indicate the possibility that a first structure within the slice is depicted or illustrated in the image of the section (e.g., it may indicate the possibility that the first structure is visible in or on the section currently milled). Similarly, the nth probability value of the original classification label may indicate the possibility that an nth structure within the slice is depicted or illustrated in the image of the section (e.g., it may indicate the possibility that the nth structure is visible in or on the section currently milled).

[0045] Now, in various aspects, a model component can electronically store, maintain, control, or otherwise access a neighborhood graph corresponding to a slice. In various cases, the neighborhood graph can include nodes and edges. In various cases, each node of the neighborhood graph can represent a corresponding one of any structures known to be contained within the slice. Thus, if a slice is known to have a total of n structures, the neighborhood graph can include n nodes: a first node representing a first structure known to be within the slice, to an nth node representing an nth structure known to be within the slice. In various aspects, the edges of the neighborhood graph can indicate which structures within the known slice are milling neighbors of each other. In other words, for any given structure within the slice, the edges of the neighborhood graph may indicate: which other structures within the slice are reachable or exposed after performing one incremental milling iteration (e.g., after removing 1 nm or 2 nm or less of material) when the given structure is known to be visible in or on a slice facet; and which other structures within the slice are not reachable or exposed after performing one incremental milling iteration when the given structure is known to be visible in or on a slice facet.

[0046] In some cases, this may be considered a type of one-way physical adjacency. As a non-limiting example, assume that structure A and structure B are positioned at different depths within the sheet, but are still physically adjacent to each other. In particular, assume that structure A is positioned at a lesser depth than structure B. In this case, structure B may be considered a milling neighbor of structure A (e.g., because one incremental milling iteration may move a cut plane from structure A down to structure B). However, in this case, structure A may not necessarily be considered a milling neighbor of structure B (e.g., if a cut plane shows structure B, this may mean that structure A, which is above structure B or at a lesser depth than structure B, has been removed via milling; therefore, one incremental milling iteration may not move a cut plane from structure B up to structure A).

[0047] In some cases, each given node in the neighborhood graph may be coupled to all other nodes in the neighborhood graph by directed edges, and the weights of those directed edges may distinguish neighboring nodes from non-neighboring nodes (e.g., lower weights (close to 0) may indicate non-neighbors, while higher weights may indicate neighbors). In some cases, each given node in the neighborhood graph may have high-weight edges to itself (e.g., a given structure may be thicker than any amount of material removed during a single incremental milling iteration; therefore, the given structure may be considered its own milling neighbor).

[0048] As a non-limiting example for purposes of illustration, assume that a known sheet includes four structures of interest (e.g., n=4): a gate; a gate inlet; a source drain; and a source drain inlet. In various aspects, the neighborhood graph may thus include four nodes: a gate node; a gate inlet node; a source drain node; and a source drain inlet node.

[0049] Now, assume that it is known that when the gate entry is visible in or on the slice, an incremental milling iteration may cause the slice to stay in the gate entry or move to the gate instead. In this case, the neighbors of the gate entry may include: itself; and the gate. Therefore, the neighborhood graph may include: a high-weight directed edge linking the gate entry node to itself; another high-weight directed edge linking the gate entry node to the gate node (e.g., in some cases, the weight of this other high-weight directed edge may be less than the weight of the recursive edge coupling the gate entry node to itself); a low-weight directed edge linking the gate entry node to the source-drain node; and another low-weight directed edge linking the gate entry node to the source-drain entry node.

[0050] Similarly, assuming it is known that when the gate is visible in or on the slice, an incremental milling iteration may cause the slice to stay within the gate or move to the source drain entry instead. In this case, the neighbors of the gate may include: itself; and the source drain entry. Thus, the neighborhood graph may include: a high-weight directed edge linking the gate node to itself; another high-weight directed edge linking the gate node to the source drain entry node (e.g., in some cases, the weight of this other high-weight directed edge may be less than the weight of the recursive edge coupling the gate node to itself); a low-weight directed edge linking the gate node to the source drain node; and another low-weight directed edge linking the gate node to the gate entry node.

[0051] Likewise, assuming that it is known that when a source drain inlet is visible in or on a cut surface, an incremental milling iteration may cause the cut surface to remain within the source drain inlet or to move to the source drain instead. In this case, the neighbors of the source drain inlet may include: itself; and the source drain. Thus, the neighborhood graph may include: a high-weight directed edge linking the source drain inlet node to itself; another high-weight directed edge linking the source drain inlet node to the source drain node (e.g., in some cases, the weight of this other high-weight directed edge may be less than the weight of the recursive edge coupling the source drain inlet node to itself); a low-weight directed edge linking the source drain inlet node to the gate node; and another low-weight directed edge linking the source drain inlet node to the gate inlet node.

[0052] Finally, assume that it is known that when the source drain is visible in or on the slice, an incremental milling iteration can cause the slice to stay within the source drain or move to another instance of the gate instead (e.g., semiconductor circuits often exhibit a repeating pattern or layout of structures). In this case, the neighbors of the source drain may include: itself; and the gate. Therefore, the neighborhood graph may include: a high-weight directed edge linking the source drain node to itself; another high-weight directed edge linking the source drain node to the gate node (e.g., in some cases, the weight of this other high-weight directed edge may be less than the weight of the recursive edge coupling the source drain node to itself); a low-weight directed edge linking the source drain node to the gate entry node; and another low-weight directed edge linking the source drain node to the source drain entry node.

[0053] In any case, the neighborhood graph can be considered to indicate which structures within a known slice are adjacent or non-adjacent to each other along a milling direction or milling axis. In various aspects, such neighborhood information can be obtained from any suitable design documentation regarding the construction or manufacture of a larger sample from which the slice was cut (e.g., can be gleaned or derived from a computer-aided design (CAD) file from which the larger sample is assumed or expected to be constructed). In various other aspects, such neighborhood information can instead be obtained from any slice that is sacrificed or destroyed to train the deep learning neural network (e.g., during manual annotation of training images captured from the sacrificed slice, the order of structures / classes along the milling direction or milling axis can be noted or recorded).

[0054] In various cases, the model component may modify the original classification labels based on the neighborhood graph to obtain modified classification labels. In various cases, such modification may depend on any one specific structure within the slice that is known or predicted to be visible in the slice or on the slice before the most recent incremental milling round or iteration. In other words, such modification may depend on any one specific structure within the slice that is indicated as visible in the slice or on the slice by a previous or previously modified classification label. Note that if no previous or previously modified classification labels are available (e.g., as may be the case on the first incremental milling round or iteration), the modification may depend on any one specific structure within the slice that is indicated as visible in the slice or on the slice by one or more previous or previously original classification labels. In any case, the model component may identify the previously visible structure (e.g., by reading or calling a previous modified classification label for the slice, or if there is no previous modified classification label for the slice, by reading or calling one or more previous original classification labels) and the neighborhood graph may indicate the neighbors and non-neighbors of the previously visible structure along the milling direction. In various aspects, the model component can modify the original classification labels by multiplying their probability values ​​by the respective weights of the neighbor and non-neighbor edges of the previously visible structure and by normalizing those resulting products to new probability values.

[0055] As a non-limiting example, assume again that the slice contains a gate, a gate inlet, a source drain, and a source drain inlet, as described above. Furthermore, assume that the original classification label is label raw =[0.05; 0.20; 0.35; 0.40], which may indicate that the source drain entry probability is 5%, the source drain probability is 20%, the gate entry probability is 35%, and the gate probability is 40%. In addition, assume that each node in the neighborhood graph has an edge weight of 100 with itself, an edge weight of 50 with each of its neighbors, and an edge weight of 0.1 with each of its non-neighbors.

[0056] Now, suppose that the previously visible structure is a source drain. In this case, the model component can modify the original classification labels by multiplication and normalization as follows. The original source drain entry probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the source drain node to the source drain entry node, resulting in a first multiplicative product: 0.05*0.1=0.005. The original source drain probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the source drain node to itself, resulting in a second multiplicative product: 0.2*100=20. The original gate entry probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the source drain node to the gate entry node, resulting in a third multiplicative product: 0.35*50=17.5. The original gate probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the source drain node to the gate node, resulting in a fourth multiplicative product: 0.40*0.1=0.04. In each case, those multiplicative products can be normalized by dividing each multiplicative product by the sum of all four multiplicative products: 0.005+20+17.5+0.04=37.545; Therefore, the modified classification label can be as follows: label modified =[0; 0.53; 0.47; 0], which may indicate a modified source drain entry probability of 0%, a modified source drain probability of 53%, a modified gate entry probability of 47%, and a modified gate probability of 0%. In other words, in this example where the previously visible structures were the source drain, the original classification label may be considered to have incorrectly indicated that the gate is now visible in or on the slice, while the modified classification label may instead be considered to more reliably indicate that the source drain is still visible in or on the slice (but the relatively high modified probability value for the gate entry may be considered to indicate that the gate entry may soon be exposed in the next milling iteration or two).

[0057] Instead, assume that the previously visible structure is a gate entry, not a source drain. In this case, the model component can modify the original classification labels by multiplication and normalization as follows. The original source drain entry probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the gate entry node to the source drain entry node, resulting in a first multiplicative product: 0.05*0.1=0.005. The original source drain probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the gate entry node to the source drain node, resulting in a second multiplicative product: 0.2*0.1=0.02. The original gate entry probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the gate entry node to itself, resulting in a third multiplicative product: 0.35*100=35. The original gate probability can be multiplied by the weight of any edge in the neighborhood graph pointing from the gate entry node to itself, resulting in a fourth multiplicative product: 0.40*50=20. In each case, those multiplication products can be normalized by dividing each multiplication product by the sum of all four multiplication products: 0.005+0.02+35+20=55.025; Therefore, the modified classification label can be as follows: label modified =[0; 0; 0.64; 36], which may indicate a modified source-drain entry probability of 0%, a modified source-drain probability of 0%, a modified gate-entry probability of 64%, and a modified gate probability of 36%. In other words, in this example where the previously visible structure is a gate entry, the original classification label may be considered to prematurely indicate that the gate is now visible in or on the cut plane, while the modified classification label may instead be considered to be more reliable or safer indicating that the gate entry is still visible in or on the cut plane (and the rather large modified probability value for the gate may be considered to indicate that the gate may soon be exposed in subsequent milling iterations).

[0058] By modifying the original classification labels in this manner according to the neighborhood graph, the model component can be considered to help enhance or otherwise improve the classification performance of the deep learning neural network. In other words, the neighborhood graph can be considered to capture or encompass a priori known structural information about the slice (e.g., indicating which structures of the slice are milling neighbors of each other), and the model component can be considered to utilize this a priori known structural information to reduce the likelihood of false positive classifications or false negative classifications from the deep learning neural network. In other words, the modification of the original classification labels with the neighborhood graph as described herein can be considered to be a type of robustness strengthening mechanism on the deep learning neural network, in which the predicted probability values ​​are enhanced for neighbors and weakened for non-neighbors. Therefore, the chance of erroneously determining that a single incremental milling iteration or wheel has moved the cut surface to a non-neighboring structure can be greatly reduced, and the chance of prematurely determining that a single incremental milling iteration or wheel has moved the cut surface to a neighboring structure can also be greatly reduced.

[0059] In any case, the modified classification labels may be considered to indicate the probability that the corresponding structure in the structure of the slice is visible in or on the cut surface. Due to the weighting / modification described herein, the modified classification labels may be considered more reliable, more confident, or more robust than the original classification labels.

[0060] Now, in various embodiments, the milling component of the computerized tool can electronically identify a target structure that is desired to be separated within the slice. In some cases, the target structure can be selected or picked by any suitable user input (e.g., a technician associated with the scientific instrument can use a keyboard or touch screen to indicate or identify the target structure). In various aspects, the milling component can determine whether the target structure is indicated by the modified classification label (e.g., whether it has the highest modified probability value among the modified classification labels).

[0061] If not, the milling assembly may conclude that the target structure is not yet visible in or on the slice facet of the lamella. Thus, the milling assembly may electronically command or direct the scientific instrument to perform another incremental milling pass or iteration, and any structures indicated by the modified classification labels may be considered or treated as previously visible structures during classification after such another incremental milling pass or iteration. That is, such another milling pass or iteration may move the slice facet slightly deeper into the lamella, the scientific instrument may capture a new image of the slice face, the deep learning neural network may generate a new original classification label for the new image, and the new original classification label may be modified via the neighborhood graph (by treating any structures indicated by the modified classification labels as previously visible structures) to derive a new modified classification label.

[0062] Conversely, if the milling component determines that the target structure is indicated by a modified classification label, the milling component can conclude that the target structure is now visible in or on the cut surface of the slice, and therefore conclude that the endpoint positioning is completed with respect to at least one side of the slice. If appropriate, the endpoint positioning procedure described above can be repeated from the reverse side of the slice. Once the target structure is visible in or on both sides of the slice (e.g., both the front and back sides of the slice), the target structure can be considered to be now physically separated in the slice (e.g., the remaining structure can be considered to have been removed via milling and is therefore no longer in the slice). Once the target structure is physically separated, the computerized tool can facilitate or initiate any other suitable analysis for the separated target structure (e.g., the separated target structure can be arranged or selected for transmission electron microscopy analysis, or such transmission electron microscopy analysis can be performed on the separated target structure).

[0063] In order to help make the classification labels described herein accurate, the deep learning neural network may first undergo training. In various aspects, the computerized tools described herein may facilitate such training in any suitable manner (e.g., in a supervised manner) based on any suitable training data set. However, it is noted that due to the implementation of the neighborhood graph described herein, the deep learning neural network may undergo much less training than the training originally required for the deep learning neural network to achieve satisfactory accuracy. In other words, it can be considered that the neighborhood graph helps to correct the false positives and false negatives generated by the deep learning neural network, and therefore there is no need to try to prevent the deep learning neural network from generating those false positives or false negatives via training on a wide range of training data sets. In other words, when the neighborhood graph is implemented as described herein, there is no need to sacrifice hundreds or thousands of slices in order to fully train the deep learning neural network.

[0064] Various embodiments described herein can be used to solve problems that are highly technical in nature (e.g., facilitating sheet endpoint localization via a graph weighted neural network), non-abstract, and cannot be performed by humans as a set of mental actions using hardware or software. In addition, some of the processes performed can be performed by a dedicated computer (e.g., a dual-beam microscope) for performing defined actions related to sheet endpoint localization.

[0065] For example, such defined actions may include: accessing, by a device operatively coupled to a processor, an image captured by a scientific instrument, wherein the image may depict a slice of a lamella; generating, by the device and via execution of a graph weighted neural network, a classification label for the slice, wherein the classification label may indicate to which of a plurality of defined classes the slice belongs; and commanding, by the device and in response to the classification label indicating that the slice does not belong to a target class of the plurality of defined classes, the scientific instrument to incrementally mill the slice of the lamella. In various aspects, the graph weighted neural network may include a deep learning neural network and a class neighborhood graph, wherein nodes of the class neighborhood graph may represent respective classes of the plurality of defined classes, and wherein weighted edges of the class neighborhood graph may indicate which of the plurality of defined classes are physically adjacent and non-adjacent to each other within the lamella along a milling direction. In various cases, the deep learning neural network may receive an image or a portion of an image as input and may produce a raw classification label as output, the raw classification label may indicate a raw probability that the slice belongs to a corresponding class among multiple defined classes, the device may multiply the raw probability by the corresponding weighted edge indicated in the class neighborhood graph and may normalize the resulting product to obtain a modified probability, and the modified probabilities may together form the classification label.

[0066] Such defined actions are inherently computerized. In fact, scientific instruments (such as dual beam microscopes) are highly technical computerized devices that include specific computerized hardware (e.g., temperature sensors, pressure sensors, voltage sensors, ion beam emitters, electron beam emitters, ion focusing lenses, mass analyzers, ion detectors, beam apertures, fluid valves). Without computers, scientific instruments and the operations performed by them cannot be realized in any reasonable or feasible way by human thinking or humans with pen and paper. In addition, the images captured by scientific instruments (e.g., electron tomography images, X-ray tomography images, confocal microscopy images) are arrays of pixels or voxels that together depict or illustrate the analysis sample or a portion thereof. Such images cannot be generated or captured in any reasonable or feasible way by human thinking or by humans with pen and paper without computers. In addition, thin slice endpoint positioning is a specific type of laboratory task that involves physically milling one or both sides of a thin slice of the analysis sample so that the thin slice contains only the desired structure or target structure. Of course, it goes without saying that neither the human mind nor a human with a pen and paper can physically mill the front and back sides of a sheet. In other words, it makes no sense to discuss the physical task of locating the end points of a sheet outside the context of computerized hardware and software.

[0067] In addition, various embodiments described herein can incorporate various teachings related to the endpoint location of a slice via a graph weighted neural network into practical applications. As explained above, it can be considered that the endpoint location of a slice is an iterative slice and image procedure in which a slice is incrementally milled, an image of the slice's section is captured, and an artificial neural network is executed on the image to determine whether the target structure or desired structure is visible in or on the section. If so, it can be considered that the endpoint location (at least about one side of the slice) is completed. If not, this milling, imaging, and classification can be repeated. In order for this process to work correctly, the artificial neural network should be trained first. Unfortunately, this training is a defective part of the prior art. In particular, some prior art trains artificial neural networks on a very large number of training data. Although this can enable the artificial neural network to achieve high classification accuracy, collecting such a large number of training data may require sacrificing or destroying hundreds or thousands of slices, which may be considered to be excessively wasteful. Other prior art trains artificial neural networks on very few training data. Although such little training data can be obtained by sacrificing or destroying only a few slices, such limited training often causes the artificial neural network to achieve unsatisfactory or insufficient classification accuracy (e.g., such little training makes the artificial neural network prone to false positives or false negatives). Therefore, the prior art can be considered disadvantageous.

[0068] Various embodiments described herein may help improve one or more of these technical problems. In particular, various embodiments described herein may facilitate slice endpoint positioning via a graph weighted neural network. More specifically, when a slice is given, the specific structures contained within the slice and the expected orientation of these specific structures within the slice along the milling direction may be known a priori (e.g., a given slice may be extracted from a larger sample constructed according to a given CAD schematic or geometry). Therefore, when a given slice is milled, the structures within the slice may be expected to be exposed, revealed, or exposed in a certain specific order known a priori (e.g., structure A is before structure B rather than after; structure C is after structure D rather than before). In various aspects, this prior knowledge may be represented in a neighborhood graph, wherein the nodes of the neighborhood graph represent corresponding structures within a given slice, and wherein the weighted edges and directed edges of the neighborhood graph represent which structures are or are not milling neighbors of each other. In various aspects, an image of a cut surface of a given thin slice can be captured, and an artificial neural network can be performed on the image or a portion thereof, thereby deriving an original classification label, which indicates the probability that the corresponding structure of the thin slice is visible in the image and therefore in the cut surface or on the cut surface. In various cases, the original classification label can be modified by the corresponding weight indicated in the neighborhood graph (e.g., via multiplication and normalization). As described herein, the inventors have experimentally verified that this can derive a modified classification label that is less likely to contain false positives or false negatives. In other words, by utilizing the prior structural knowledge of the thin slice captured by the neighborhood graph, the classification accuracy of the artificial neural network can be improved. Therefore, a satisfactory or sufficient level of classification accuracy can be obtained without sacrificing or destroying many thin slices. In fact, various experiments conducted by the inventors show that a satisfactory or sufficient classification accuracy can be obtained by sacrificing only six thin slices. Therefore, it can be considered that the various embodiments described herein solve or improve various technical problems or shortcomings that plague the prior art. At least for these reasons, it can be considered that the various embodiments described herein are specific and practical technical improvements in the field of thin slice endpoint positioning. Therefore, the various embodiments described herein certainly qualify as useful and practical applications of computers.

[0069] Furthermore, various embodiments described herein may control real-world tangible devices based on the disclosed teachings. For example, various embodiments described herein may electronically activate, deactivate, or otherwise actuate real-world hardware (e.g., ion beam emitters, ion focusing lenses, carrier fluid valves / pumps) of real-world scientific instruments (e.g., dual-beam microscopes), perform real-world analysis on real-world data captured by those real-world scientific instruments (e.g., generate classification labels for images captured by the scientific instruments), and may electronically initiate real-world actions based on the results of such real-world analysis (e.g., a slice may be incrementally milled based on classification labels generated for a slice's facets).

[0070] Figure 1 An example non-limiting block diagram of a scientific instrument module 102 is illustrated in accordance with various embodiments described herein.

[0071] In various embodiments, the scientific instrument module 102 may be implemented by circuitry (e.g., including electrical or optical components) such as a programmed computing device. The logical components of the scientific instrument module 102 may be included in a single computing device, or may be distributed across multiple computing devices that communicate with each other as appropriate. Fig. 22 and Fig.24 Examples of computing devices that may implement the scientific instrument module 102, either alone or in combination, are discussed and reference is made to Fig.23 and Fig.25 Examples of systems or networks of interconnected computing devices that may implement the scientific instrument module 102 across one or more of the computing devices are discussed.

[0072] The scientific instrument module 102 may include a first logic component 104, a second logic component 106, and a third logic component 108. As used herein, the term "logic component" may include a device that performs a set of operations associated with logic. For example, any of the logic elements included in the scientific instrument module 102 may be implemented by one or more computing devices, which are programmed with instructions to cause one or more processing devices of the computing device to perform an associated set of operations. In a specific embodiment, the logic element may include one or more non-transient computer-readable media, which have instructions on them, which, when executed by one or more processing devices in one or more computing devices, cause the one or more computing devices to perform an associated set of operations. As used herein, the term "module" may refer to a collection of one or more logic elements, which together perform functions associated with the module. Different logic elements in the logic elements in the module may take the same form or may take different forms. For example, some logic components in the module may be implemented by a programmed general-purpose processing device, while other logic components in the module may be implemented by an application-specific integrated circuit (ASIC). As another example, different logic elements in the logic elements in the module may be associated with different instruction sets executed by one or more processing devices. A module may omit one or more logic elements shown in the associated figures; for example, when a module is to perform a subset of the operations discussed herein with reference to the module, the module may include a subset of the logic elements depicted in the associated figures.

[0073] In various embodiments, there may be a scientific instrument corresponding to the scientific instrument module 102. In various aspects, the scientific instrument can be any suitable computerized device that can electronically measure some scientifically relevant, clinically relevant or research-related characteristics, properties or attributes of (e.g., a known or unknown mixture, compound or set of substances) an analytical sample. As a non-limiting example, the scientific instrument can be a mass spectrometer operatively coupled to a gas chromatograph or a liquid chromatograph. In this case, the scientific instrument can measure or determine the ion spectrum (e.g., relative ion abundance as a function of mass-to-charge ratio) of the analytical sample. As another non-limiting example, the scientific instrument can be a scanning electron microscope. In this case, the scientific instrument can measure or determine the surface morphology of the analytical sample. As another non-limiting example, the scientific instrument can be a transmission electron microscope. In this case, the scientific instrument can measure or determine the internal structural details of the analytical sample. As a more general non-limiting example, the scientific instrument can be any suitable type of charged particle microscope (e.g., some types of microscopes can use non-electron ion beams to capture images).

[0074] In some cases, the scientific instrument may be capable of imaging and milling thin slices cut or extracted from the analysis sample. As a non-limiting example, the scientific instrument may be a dual beam microscope equipped with both an ion beam emitter and an electron beam emitter. In this case, the ion beam emitter may facilitate milling (e.g., material removal) of the thin slice at various voltages, currents, or power levels, and the electron beam emitter may facilitate imaging of the thin slice (in conjunction with an electron detector and an optical lens).

[0075] In various embodiments, the first logic component 104 can access an image captured or otherwise generated by a scientific instrument. In various aspects, the image can be any suitable pixel array or voxel array that depicts or illustrates a slice (eg, a milling site).

[0076] In various embodiments, the second logic component 106 may generate a classification label for the slice via execution of the graph weighted neural network, wherein the classification label may indicate which of the multiple defined classes the slice belongs to. More specifically, the multiple defined classes may represent multiple structures known to be contained in or somewhere inside the slice, respectively, and thus the classification label may indicate the probability that a corresponding structure of the multiple structures is visible in or on the slice of the slice. In various aspects, the graph weighted neural network may include a deep learning neural network and a class neighborhood graph. In various cases, the deep learning neural network may be configured as an image classifier. Therefore, the logic component 106 may involve feeding the image (or any suitable portion thereof) as input to the deep learning neural network, so that the deep learning neural network produces the original classification label as output. In various cases, the class neighborhood graph may include nodes and edges. In various aspects, the nodes may represent the corresponding classes and therefore the corresponding structures of the slice. In various cases, the edges may represent or indicate which nodes and therefore which structures of the slice are or are not milling neighbors of each other. In various cases, the logic component 106 may involve modifying the original classification labels with the corresponding weights indicated in the class neighborhood graph via multiplication and normalization to convert the original classification labels into classification labels. As explained herein, such modification based on the class neighborhood graph may be considered to utilize prior knowledge of the slices in order to improve or enhance the classification accuracy of the deep learning neural network.

[0077] In various embodiments, the third logic component 108 may command the scientific instrument to incrementally mill a slice of the thin film in response to the classification label indicating that the slice does not belong to a target class among multiple defined classes (e.g., in response to determining that the target structure is not yet visible in or on the slice of the thin film).

[0078] In various aspects, the imaging operations, graph weighted classification operations, and milling operations covered by the first logic component 104, the second logic component 106, and the third logic component 108 may be repeated until the most recent classification label indicates that the slice belongs to the target class (e.g., indicating that the target structure is now visible in or on the slice of the lamella). At this point, the endpoint positioning of the lamella may be considered complete with respect to at least one side of the lamella. If appropriate, this procedure may be repeated for the opposite or reverse side of the lamella, such that the target structure becomes physically separated (e.g., such that sufficient material is removed from the lamella so that only the target structure remains within the lamella).

[0079] Therefore, the scientific instrument module 102 may facilitate sheet endpoint localization via a graph weighted neural network.

[0080] Figure 2 is an example non-limiting flowchart of a computer-implemented method 200 according to various embodiments described herein. The operations of the computer-implemented method 200 may be used in any suitable context to perform any suitable operations (e.g., may be performed by a computer program). Figure 1 , Fig.21 , Fig. 22 , Fig.23 , Fig.24 and Fig.25 Any of the various modules, computing devices, or graphical user interfaces described herein may be executed or used in conjunction with them). Figure 2 In the description, operations are each illustrated once in a particular order, but operations may be reordered or repeated as needed and appropriate (eg, different operations may be performed in parallel where appropriate).

[0081] In various aspects, act 202 may include accessing, by a device operatively coupled to a processor, an image captured by a scientific instrument, wherein the image may depict a cut surface of a slice. In various cases, first logic component 104 may perform or otherwise facilitate act 202 .

[0082] In various cases, action 204 may include performing, by the device and via execution of the graph weighted neural network, a second operation of generating a classification label for the cut surface, wherein the classification label may indicate which of a plurality of defined classes the cut surface belongs to. In various cases, second logic component 106 may perform or otherwise facilitate action 204.

[0083] In various aspects, act 206 may include commanding, by the device and in response to the classification tag indicating that the slice does not belong to a target class among the plurality of defined classes, the scientific instrument to incrementally mill the slice. In various cases, third logic component 108 may perform or otherwise facilitate act 206 .

[0084] Thus, the computer-implemented method 200 may facilitate sheet endpoint localization via a graph weighted neural network.

[0085] Figure 3 Illustrated is a block diagram of an example non-limiting system that can facilitate sheet endpoint localization via a graph weighted neural network in accordance with one or more embodiments described herein.

[0086] In various embodiments, there may be a scientific instrument 302. In various aspects, the scientific instrument 302 may be as described above. That is, the scientific instrument 302 may be any suitable computerized device that can electronically measure any suitable scientifically relevant, clinically relevant, or research-related characteristics, attributes, or properties of any suitable analytical sample. In particular, the scientific instrument 302 may be any suitable computerized device that can utilize its constituent hardware (e.g., electron source, anode, condenser lens, condenser aperture, scanning coil, objective lens, objective aperture, deflector, condenser, stigmator, electron detector, X-ray detector, actuatable sample stage) to facilitate both imaging and milling of the thin slice 304. As a non-limiting example, the scientific instrument 302 may be a dual-beam charged particle microscope.

[0087] In various aspects, the thin slice 304 may be cut or otherwise extracted from any suitable analytical sample. In various cases, the thin slice 304 may be any suitable rectangular plate, layer or slice of the analytical sample, so that the thickness of the thin slice 304 may be one or more times or one or more orders of magnitude smaller than the length and width of the thin slice. As a non-limiting example, the thickness of the thin slice 304 may be in the range of 50nm to 130nm, while the length and width of the thin slice 304 may instead be measured as approximately several microns each. In any case, the thin slice 304 may physically include a plurality of structures 305. In various aspects, for any suitable positive integer n, the plurality of structures 305 may include n structures: structure 305(1) to structure 305(n). In various cases, each of the plurality of structures 305 may be any suitable designed or constructed structure that can be manufactured in or on the analytical sample according to any suitable photolithographic technique. As some non-limiting examples, any of the plurality of structures 305 may be: a transistor gate; an entry region or an exit region of a transistor gate; a transistor source drain; an entry region or an exit region of a transistor source drain; a transistor fin; an entry region or an exit region of a transistor fin; any other suitable micro-electromechanical circuit component; or an entry region, an exit region, or any other suitable region of any other suitable micro-electromechanical circuit component. In various cases, the process or procedure of thin-sheet endpoint positioning may be considered to be the first task performed in order to check whether any or all of the plurality of structures 305 are appropriately or correctly constructed.

[0088] In various aspects, scientific instrument 302 can electronically capture or generate image 306 about slice 304. In some cases, image 306 can be a two-dimensional array of pixels. In other cases, image 306 can instead be a three-dimensional array of voxels. In any case, image 306 can visually depict or illustrate a slice 308 of slice 304. In various aspects, slice 308 can be a surface of slice 304 on which scientific instrument 302 has performed a milling operation. In other words, slice 308 can be a surface of slice 304 from which scientific instrument 302 has removed material (e.g., via emission of a high-pressure or low-pressure ion beam).

[0089] As a non-limiting example, assume that the length, width, and thickness of the slice 304 extend along the x-axis, y-axis, and z-axis, respectively, of Cartesian three-dimensional space. In this case, assuming that milling is performed in the negative z-direction (e.g., in a top-down direction), the cut surface 308 can be considered to be the topmost surface of the slice 304, which can be parallel to the xy plane. Equivalently, if milling is performed in the positive z-direction (e.g., in a bottom-up direction) instead, the cut surface 308 can be considered to be the bottommost surface of the slice 304, which can be parallel to the xy plane. In any case, the cut surface 308 can be considered to be the plane or surface of the slice 304 from which material is removed via milling, such that such a plane or surface can be substantially parallel to the xy plane and can be located at a certain depth along the z-axis. Note that each progressive or continuous milling wheel or iteration can be considered to move the cut surface 308 to a new depth along the z-axis. For example, if milling is performed in the negative z-direction (e.g., in a top-down direction), the slice 308 may start as the uppermost or topmost surface of the slice 304 and may move to a lower depth or position with each milling wheel along the z-axis. As another example, if milling is performed in the positive z-direction (e.g., in a bottom-up direction), the slice 308 may start as the lowermost or bottommost surface of the slice 304 and may move to a higher depth or position with each milling wheel along the z-axis. In any case, the slice 308 may be any surface of the slice 304 from which material has been removed by the scientific instrument 302 via milling. Thus, the slice 308 may be considered to be an internal cross-section of the slice 304 that is exposed, revealed, or exposed via milling.

[0090] In various cases, system 310 may be electronically integrated with scientific instrument 302 or with image 306 (e.g., via any suitable wired or wireless electronic connection). In various cases, it may be desirable to mill lamina 304 until a particular one of plurality of structures 305 is visible in or on slice 308. Indeed, it may be desirable to perform such milling from both the top and bottom sides of lamina 304 in order to physically separate the particular one of plurality of structures 305 in lamina 304. As described herein, system 310 may help achieve this goal.

[0091] In various aspects, the system 310 may include a processor 312 (e.g., a computer processing unit, a microprocessor) and a non-transitory computer-readable memory 314 operatively or operatively or communicatively connected or coupled to the processor 312. The non-transitory computer-readable memory 314 may store computer-executable instructions that, when executed by the processor 312, may cause the processor 312 or other components of the system 310 (e.g., an access component 316, a model component 318, a milling component 320) to perform one or more actions. In various embodiments, the non-transitory computer-readable memory 314 may store computer-executable components (e.g., an access component 316, a model component 318, a milling component 320), and the processor 312 may execute these computer-executable components.

[0092] In various embodiments, the system 310 may include an access component 316. In various aspects, the access component 316 may electronically access the scientific instrument 302. That is, the access component 316 may electronically communicate with or otherwise electronically interact with the scientific instrument 302 (e.g., transmit electronic instructions or commands to the scientific instrument, receive electronic data from the scientific instrument). Therefore, the access component 316 may be considered an agent or channel through which other components of the system 310 may interact, communicate with, or otherwise manipulate the scientific instrument 302. In various cases, the access component 316 may electronically access the image 306. That is, the access component 316 may electronically receive, electronically retrieve, or otherwise electronically obtain the image 306 from any suitable electronic source or database (not shown). As a non-limiting example, the access component 316 may electronically receive, retrieve, or otherwise obtain the image 306 from the scientific instrument 302. In any case, access component 316 can be considered as an agent or channel through which other components of system 310 can interact with image 306 or otherwise manipulate the image. However, these are only non-limiting examples. In other cases, access component 316 can be omitted, and any other component of system 310 can communicate or interact directly with scientific instrument 302 or with image 306.

[0093] In various embodiments, system 310 may include model component 318. In various aspects, as described herein, model component 318 may generate a classification label for slice 308 by performing a graph weighted neural network on image 306. In various cases, the classification label may indicate a probability that a corresponding one of plurality of structures 305 is visible in or on slice 308.

[0094] In various embodiments, system 310 can include milling assembly 320. In various cases, as described herein, milling assembly 320 can command scientific instrument 302 to perform additional or incremental milling of slice 304 in response to the classification tag indicating that a target or desired one of plurality of structures 305 is not yet visible in or on slice 308.

[0095] Note that in various cases, the access component 316, the model component 318, and the milling component 320 may be considered collectively to be one or more software components 315 of the system 310. In various aspects, it should be appreciated that for ease of explanation and illustration, the one or more software components 315 are primarily described herein as including three components (e.g., the access component 316, the model component 318, and the milling component 320). However, the one or more software components 315 are not limited to being implemented as exactly such three components in each embodiment. In fact, in some embodiments, the functionality described herein of such three components may be combined in any suitable manner so as to be implemented in or by less than three components (e.g., in some cases, a single component may perform all of the functionality described herein with respect to the access component 316, the model component 318, and the milling component 320). In other embodiments, the functionality described herein of such three components may instead be distributed, separated, split, or segmented in any suitable manner so as to be implemented in or by more than three components (e.g., two or more components may facilitate functionality that may be performed by access component 316; two or more components may facilitate functionality that may be performed by model component 318; two or more components may facilitate functionality that may be performed by milling component 320).

[0096] Figure 4 Illustrated is a block diagram of an example non-limiting system including a graph weighted neural network and classification labels that can facilitate slice endpoint localization via the graph weighted neural network in accordance with one or more embodiments described herein.

[0097] In various embodiments, the model component 318 can electronically store, electronically maintain, electronically control, or otherwise electronically access the graph weighted neural network 402. In various aspects, the model component 318 can electronically execute the graph weighted neural network 402 on the image 306 (or a portion thereof). In various cases, such execution can cause the graph weighted neural network 402 to generate the classification label 404. Figures 5 to 9 Non-limiting aspects are described.

[0098] Figures 5 to 9 An example non-limiting block diagram showing how a graph weighted neural network 402 may be used to generate classification labels 404 is illustrated in accordance with one or more embodiments described herein.

[0099] First, consider Figure 5 In various aspects, the graph weighted neural network 402 may include a deep learning neural network 502 and a class neighborhood graph 504 .

[0100] In various cases, the deep learning neural network 502 may have or otherwise present any suitable deep learning internal architecture. For example, the deep learning neural network 502 may have an input layer, one or more hidden layers, and an output layer. In various cases, any of such layers may be coupled together by any suitable inter-neuron connection or inter-layer connection, such as a forward connection, a skip connection, or a recurrent connection. In addition, in various cases, any of such layers may be a neural network layer of any suitable type with any suitable learnable or trainable internal parameters. For example, any of such input layer, one or more hidden layers, or output layer may be a convolutional layer, and the learnable or trainable parameters of the convolutional layer may be a convolution kernel. As another example, any of such input layer, one or more hidden layers, or output layer may be a dense layer, and its learnable or trainable parameters may be a weight matrix or a bias value. As yet another example, any of such input layer, one or more hidden layers, or output layer may be a batch normalization layer, and its learnable or trainable parameters may be a shift factor or a scale factor. As yet another example, any of such input layer, one or more hidden layers, or output layer may be an LSTM layer, and the learnable or trainable parameters of these LSTM layers may be an input state weight matrix or a hidden state weight matrix. In addition, in various cases, any of such layers may be a neural network layer of any suitable type with any suitable fixed or non-trainable internal parameters. For example, any of such input layer, one or more hidden layers, or output layer may be a non-linear layer, a padding layer, a pooling layer, or a cascade layer. Regardless of the specific internal architecture of the deep learning neural network 502, the deep learning neural network may be configured as an image classifier.

[0101] In various aspects, class neighborhood graph 504 may be a graph data structure including nodes and edges. In various cases, the nodes of class neighborhood graph 504 may represent multiple classes 506, respectively. In various cases, multiple classes 506 may correspond to multiple structures 305, respectively. Therefore, since multiple structures 305 may include n structures, multiple classes 506 may include n classes: class 506(1) to class 506(n). In various aspects, each of multiple classes 506 may be considered to be a defined classification category indicating or representing a corresponding one of multiple structures 305. As a non-limiting example, class 506(1) may correspond to and therefore represent structure 305(1). That is, class neighborhood graph 504 may include a first node representing class 506(1) and therefore representing structure 305(1). As another non-limiting example, class 506(n) may correspond to and therefore represent structure 305(n). Thus, the class neighborhood graph 504 may include an nth node representing the class 506 ( n ) and, therefore, the structure 305 ( n ).

[0102] Now, as mentioned above, each of the plurality of structures 305 may be physically located somewhere within or inside the thin slice 304. In various aspects, it may be known a priori how the plurality of structures 305 are physically ordered or arranged within or inside the thin slice 304 along any axis or direction in which milling is performed by the scientific instrument 302 (or at least, how the plurality of structures are assumed or expected to be physically ordered or arranged). In fact, as mentioned above, the thin slice 304 may be cut or extracted from an analysis sample, and the analysis sample may be manufactured or constructed (e.g., via photolithography) according to any suitable engineering design. In some cases, such an engineering design may be represented or communicated by a three-dimensional CAD illustration or model of the analysis sample. In any case, it may be known which of the plurality of structures 305 are expected to be adjacent to each other along a milling direction or milling axis, or which of the plurality of structures are expected to be non-adjacent to each other along a milling direction or milling axis. In other words, if an incremental milling wheel or iteration is to be performed starting from any given one of the plurality of structures 305 (e.g., removing 1 nm or 2 nm or less of material), it may be known which others of the plurality of structures 305 such incremental milling wheel or iteration may and may not uncover, expose, or reveal. For ease of explanation, if an incremental milling wheel or iteration starting at a first structure may uncover, expose, or reveal a second structure, the second structure may be referred to as a milling neighbor of the first structure (e.g., the first structure and the second structure may be separated from each other in the milling direction by an amount of material that is thin enough to be passed over or removed by a single incremental milling wheel or iteration). In contrast, if an incremental milling wheel or iteration starting at a first structure may not uncover, expose, or reveal the second structure, the second structure may be considered not to be a milling neighbor of the first structure (e.g., the first structure and the second structure may be separated from each other in the milling direction by an amount of material that is thick enough to be passed over or removed by a single incremental milling wheel or iteration).

[0103] Note that the milling neighbor relationship may be considered unidirectional or otherwise considered not necessarily bidirectional. In fact, if the second structure is a milling neighbor of the first structure, this may mean that the second structure may be uncovered, exposed or revealed by a single incremental milling wheel or iteration starting somewhere in the first structure. In other words, the second structure may be considered to be located or positioned deeper in the sheet 304 along the milling direction or milling axis than the first structure. Therefore, although the first structure and the second structure may be physically adjacent to each other along the milling direction, the first structure may be removed by milling in a chronological order before removing the second structure. Therefore, if milling is subsequently performed starting somewhere in the second structure, such milling cannot uncover, expose or reveal the first structure because the first structure has already been removed via the previous milling. Therefore, even though it may be the case that the second structure is a milling neighbor of the first structure, it is not necessarily the case that the first structure is a milling neighbor of the second structure. However, in some cases, it may still be possible that a first structure is a milled neighbor of a second structure, and a second structure is a milled neighbor of the first structure (eg, the second structure may be sandwiched between two instances of the first structure in sheet 304).

[0104] In various cases, the edges of class neighborhood graph 504 may indicate or communicate which classes of plurality of classes 506, and therefore which structures of plurality of structures 305, are or are not milling neighbors of each other. In particular, each node of class neighborhood graph 504 may have a high-weight directed edge with itself, may have one or more medium-weight directed edges (also referred to as medium-weight directed edges) with any node of class neighborhood graph 504 that is a milling neighbor of the node, and may have one or more low-weight directed edges with any node of class neighborhood graph 504 that is not a milling neighbor of the node. In this manner, edge weights may be considered to indicate milling adjacency or milling neighbor relationships.

[0105] As a non-limiting example, class 506(1), and therefore structure 305(1), may be represented by a first node of class neighborhood graph 504. In various aspects, the first node may have a high-weight directed edge 508(1), one or more medium-weight directed edges 510(1), and one or more low-weight directed edges 512(1).

[0106] In various cases, high-weight directed edge 508(1) may point from the first node to the first node. In other words, high-weight directed edge 508(1) may be considered a recursive edge that the first node has with itself. In various cases, high-weight directed edge 508(1) may be realized because class 506(1), and therefore structure 305(1), may be considered its own milling neighbor. In fact, structure 305(1) may be thick enough that an incremental milling turn or iteration starting somewhere within structure 305(1) may uncover, reveal, or expose some deeper portion of structure 305(1).

[0107] In various aspects, each of the one or more medium-weight directed edges 510(1) may point from the first node to some other corresponding node of the class neighborhood graph 504, where the other corresponding node may represent a corresponding milling neighbor of the class 506(1) and, therefore, the structure 305(1). Assume that for any suitable positive integer i, there are i classes and, therefore, i structures that are known or expected to be milling neighbors of the class 506(1) and, therefore, the structure 305(1). In this case, the one or more medium-weight directed edges 510(1) may include a total of i medium-weight directed edges: one medium-weight directed edge for each milling neighbor of the class 506(1) (e.g., one medium-weight directed edge for each milling neighbor of the structure 305(1)).

[0108] In various cases, each of the one or more low-weight directed edges 512(1) may point from the first node to some other corresponding node of the class neighborhood graph 504, where the other corresponding node may represent a corresponding non-milled neighbor of the class 506(1) and, therefore, the structure 305(1). Assume that for any suitable positive integer j, there are j classes and, therefore, j structures that are known or expected not to be milled neighbors of the class 506(1) and, therefore, the structure 305(1). In this case, the one or more low-weight directed edges 512(1) may include a total of j low-weight directed edges: one low-weight directed edge for each non-milled neighbor of the class 506(1) (e.g., one low-weight directed edge for each non-milled neighbor of the structure 305(1)).

[0109] In various aspects, the high-weight directed edge 508(1) may be assigned a first scalar weight In various cases, each of the one or more medium-weight directed edges 510(1) may be assigned a second scalar weight w mid ∈R + (e.g., so that all of the one or more medium-weight directed edges 510(1) have the same weight as one another.) In various cases, each of the one or more low-weight directed edges 512(1) may be assigned a third scalar weight w low ∈R + (e.g., so that all one or more low-weight directed edges 512(1) have the same weight as each other). In various aspects, the situation may be w low <w mid <w high , hence the terms "high weight", "medium weight", and "low weight". For example, the case could be w low =0.1, w mid =50.0 and w high =100.0.

[0110] As another non-limiting example, the class 506(n), and therefore the structure 305(n), may be represented by the nth node of the class neighborhood graph 504. In various aspects, the nth node may have a high-weight directed edge 508(n), one or more medium-weight directed edges 510(n), and one or more low-weight directed edges 512(n).

[0111] As described above, the high-weight directed edge 508(n) may point from the nth node to the nth node, such that the high-weight directed edge 508(n) may be considered a recursive edge that the nth node has with itself. Again, this may be due to the fact that the class 506(n), and therefore the structure 305(n), may be considered to be its own milling neighbor (e.g., the structure 305(n) may be thick enough that an incremental milling wheel or iteration starting somewhere within the structure 305(n) may uncover, reveal, or expose some deeper portion of the structure 305(n)).

[0112] As described above, each of the one or more medium-weight directed edges 510n may point from the nth node to some other corresponding node of the class neighborhood graph 504, where the other corresponding node may represent a corresponding milling neighbor of the class 506(n) and, therefore, the structure 305(n). Thus, if, for any suitable positive integer k, there are k classes and, therefore, k structures that are known or expected to be milling neighbors of the class 506(n) and, therefore, the structure 305(n), then the one or more medium-weight directed edges 510(n) may include a total of k medium-weight directed edges: one medium-weight directed edge for each milling neighbor of the class 506(n) (e.g., one medium-weight directed edge for each milling neighbor of the structure 305(n)).

[0113] Similar to the above, each of the one or more low-weight directed edges 512(n) may point from the first node to some other corresponding node of the class neighborhood graph 504, where the other corresponding node may represent a corresponding non-milled neighbor of the class 506(n) and, therefore, the structure 305(n). Thus, if for any suitable positive integer 1, there is 1 class and, therefore, 1 structure that is known or expected to be a milled neighbor of the class 506(n) and, therefore, the structure 305(n), then the one or more low-weight directed edges 512(n) may include a total of 1 low-weight directed edge: one low-weight directed edge for each non-milled neighbor of the class 506(n) (e.g., one low-weight directed edge for each non-milled neighbor of the structure 305(n)).

[0114] In various aspects, the high-weight directed edge 508n may be assigned a first scalar weight w high (e.g., the same weight as high-weight directed edge 508(1)). In various cases, each of the one or more medium-weight directed edges 510(n) may be assigned a second scalar weight w mid(e.g., the same weight as assigned to each of the one or more medium-weight directed edges 510(1)). In various cases, each of the one or more low-weight directed edges 512(n) may be assigned a third scalar weight w low (eg, the same weight as each of the one or more low-weight directed edges 512(1)).

[0115] In various aspects, high-weight directed edges 508(1) through high-weight directed edges 508(n) may be collectively referred to as a plurality of high-weight directed edges 508. In various instances, one or more medium-weight directed edges 510(1) through one or more medium-weight directed edges 510(n) may be collectively referred to as a plurality of medium-weight directed edges 510. In various instances, one or more low-weight directed edges 512(1) through one or more low-weight directed edges 512(n) may be collectively referred to as a plurality of low-weight directed edges 512.

[0116] It should be appreciated that the specific edge weights or edge weight relationships described herein are merely non-limiting examples. In various cases, any edges within class neighborhood graph 504 may have the same or different weights from one another.

[0117] Now, consider Figure 6 . In various embodiments, as mentioned above, the deep learning neural network 502 can be configured as an image classifier. Thus, in various aspects, the model component 318 can electronically execute the deep learning neural network 502 on the image 306. In various cases, such execution can cause the deep learning neural network 502 to generate the original classification label 602. More specifically, the model component 318 can feed the image 306 to the input layer of the deep learning neural network 502. In various aspects, the image 306 can complete a forward pass through one or more hidden layers of the deep learning neural network 502. In various cases, the output layer of the deep learning neural network 502 can calculate or operate on the original classification label 602 based on the activation map or feature map generated by the one or more hidden layers.

[0118] In some cases, deep learning neural network 502 may be configured to receive the entire image 306 as input. However, this is merely a non-limiting example. In other cases, deep learning neural network 502 may instead be configured to receive any suitable portion of image 306 as input. As a non-limiting example, deep learning neural network 502 may be configured to receive as input a device line that may be extracted from image 306. In particular, a device line may be considered to be a horizontal sub-portion of image 306 that depicts an area or region of interest of slice 308. For example, if image 306 is a 600 by 600 pixel array (e.g., 600 rows of pixels by 600 columns of pixels), a device line may be a 100 by 600 pixel array (e.g., 100 consecutive rows of pixels that may be cropped from image 306 and that are considered or expected to depict or show the most interesting area or region of slice 308). In various aspects, device lines may be extracted via any suitable image processing technique. For example, the image 306 may be decomposed into a plurality of regular vertical strips (e.g., each vertical strip may include one or more columns of pixels), and a sliding window of constant size may be moved down each vertical strip, one or more rows of pixels at a time. At each position of the sliding window, a standard deviation of intensity values ​​may be computed for any pixel within the sliding window at that position. For each vertical strip, there may be a position of the sliding window for which the computed standard deviation increases by more than any suitable threshold margin, and the position may be considered to depict the top boundary of the device line in the vertical strip. By performing this sliding window technique for a plurality of vertical strips, the entire top boundary of the device line may be obtained (e.g., via extrapolation of a best fit line through those identified positions of the sliding window). In this way, the device line may be identified and thus extracted or cropped from the image 306, and the model component 318 may execute the deep learning neural network 502 on the device line rather than on the entire image 306. That is, the model component 318 can feed the device line to the input layer of the deep learning neural network 502, the device line can complete the forward pass through one or more hidden layers of the deep learning neural network 502, and the output layer of the deep learning neural network 502 can calculate or operate the original classification label 602 based on the activation map or feature map generated by the one or more hidden layers.

[0119] In any case, the original classification labels 602 may include a plurality of original class probabilities 604. In various aspects, the plurality of original class probabilities 604 may correspond to the plurality of classes 506 and, therefore, to the plurality of structures 305, respectively. Thus, because the plurality of classes 506 may include n classes, and because the plurality of structures 305 may include n structures, the plurality of original class probabilities 604 may include n probabilities: original class probability 604(1) to original class probability 604(n). In various cases, each of the plurality of original class probabilities 604 may be a scalar indicating an inferred or predicted probability or likelihood that the slice 308 belongs to a corresponding one of the plurality of classes 506. In other words, each of the plurality of original class probabilities 604 may be a scalar indicating an inferred or predicted probability or likelihood that a corresponding one of the plurality of structures 305 is currently visible in or on the slice 308.

[0120] As a non-limiting example, raw class probability 604(1) may correspond to class 506(1) and, therefore, structure 305(1). Thus, raw class probability 604(1) may be a real-valued positive scalar whose magnitude indicates the likelihood that slice 308 depicted by image 306 belongs to class 506(1) (as determined or predicted by deep learning neural network 502). In other words, raw class probability 604(1) may be a real-valued positive scalar whose magnitude indicates the likelihood that structure 305(1) is currently visible, revealed, exposed, or otherwise exposed in or on slice 308 (as determined or predicted by deep learning neural network 502).

[0121] As another non-limiting example, the raw class probability 604(n) may correspond to the class 506(n) and, therefore, to the structure 305(n). Thus, the raw class probability 604(n) may be a real-valued positive scalar whose magnitude indicates the likelihood that the slice 308 depicted by the image 306 belongs to the class 506(n) (as determined or predicted by the deep learning neural network 502). In other words, the raw class probability 604(n) may be a real-valued positive scalar whose magnitude indicates the likelihood that the structure 305(n) is currently visible, revealed, exposed, or otherwise exposed in or on the slice 308 (as determined or predicted by the deep learning neural network 502).

[0122] In various cases, note that the plurality of original class probabilities 604 may sum to one. That is, the sum of the plurality of original class probabilities 604 may equal one.

[0123] Now, consider Figure 7 In various embodiments, the model component 318 can electronically modify the original classification label 602 based on the class neighborhood graph 504. In various aspects, such modification can result in the classification label 404.

[0124] More specifically, there may be a base class 702. In various cases, the base class 702 may be any one of the plurality of classes 506. In particular, there may be a previous image (not shown) captured by the scientific instrument 302, wherein the previous image may depict or illustrate a previous version of the slice 308. In other words, the previous image may show whatever the slice 308 looked like just before or before the most recent milling pass or iteration performed by the scientific instrument 302. In various aspects, the system 310 may analyze the previous image as described herein, and the base class 702 may be any one of the plurality of classes 506 to which the system 310 determines or predicts that the previous version of the slice 308 belongs. In other words, the base class 702 may correspond to any one of the plurality of structures 305 that is determined, predicted, or inferred to have been visible in or on the slice 308 just before the most recent milling pass or iteration. In some cases, base class 702 may be any one of multiple classes 506 indicated by a previous classification label generated by model component 318 via modification of class neighborhood graph 504. However, if no such previous classification label exists (e.g., as is the case during a first iteration during which class neighborhood graph 504 is utilized), base class 702 may instead be any one of multiple classes 506 indicated by one or more previous original classification labels generated by model component 318.

[0125] In any case, the class neighborhood graph 504 may indicate edges branching off or pointing away from the base class 702. In particular, the class neighborhood graph 504 may indicate that the base class 702 has a high-weight directed edge 704 with itself (e.g., the high-weight directed edge 704 may come from the plurality of high-weight directed edges 508). In addition, the class neighborhood graph 504 may indicate that the base class 702 has one or more medium-weight directed edges 706, and any class, and therefore any structure, is known or expected to be a milling neighbor of the base class 702 (e.g., the one or more medium-weight directed edges 706 may come from the plurality of medium-weight directed edges 510). In addition, the class neighborhood graph 504 may indicate that the base class 702 has one or more low-weight directed edges 708, and any class, and therefore any structure, is known or expected not to be a milling neighbor of the base class 702 (e.g., the one or more low-weight directed edges 708 may come from the plurality of low-weight directed edges 512).

[0126] In various aspects, the model component 318 can modify the original classification labels 602 by multiplying the plurality of original class probabilities 604 with respective weights or corresponding weights from the high weight directed edge 704 , from one or more medium weight directed edges 706 , and from one or more low weight directed edges 708 .

[0127] As a non-limiting example, consider the original class probability 604(1). As mentioned above, the original class probability 604(1) may correspond to class 506(1). In various aspects, the model component 318 may calculate a first product by multiplying the original class probability 604(1) by any weight corresponding to class 506(1) from the high-weight directed edge 704, from one or more medium-weight directed edges 706, or from one or more low-weight directed edges 708. In other words, the model component 318 may calculate a first product by multiplying the original class probability 604(1) by the weight of any edge in the class neighborhood graph 504 that points away from the base class 702 and toward class 506(1). For example, assume that class 506(1) is the same as the base class 702. In this case, the high-weight directed edge 704 may point away from the base class 702 and toward class 506(1). Therefore, the model component 318 may multiply the original class probability 604(1) by the weight w high , resulting in a first product. As another example, assume that class 506(1) is different from base class 702 and is known or expected to be a milling neighbor of base class 702. In this case, one of the one or more medium-weight directed edges 706 may point away from base class 702 and toward class 506(1). Therefore, model component 318 may multiply raw class probability 604(1) by weight w mid , resulting in the first product. As yet another example, assume that class 506(1) is different from base class 702 and is known or expected not to be a milling neighbor of base class 702. In this case, one of the one or more low-weight directed edges 708 may point away from base class 702 and toward class 506(1). Therefore, model component 318 may multiply raw class probability 604(1) by weight w low , thus obtaining the first product.

[0128] As another non-limiting example, consider the raw class probabilities 604(n). As mentioned above, the raw class probabilities 604(n) may correspond to the class 506(n). In various aspects, the model component 318 may compute the nth product by multiplying the raw class probabilities 604(n) by any one of the weights corresponding to the class 506(n) from the high-weight directed edges 704, from one or more of the medium-weight directed edges 706, or from one or more of the low-weight directed edges 708. In other words, the model component 318 may compute the nth product by multiplying the raw class probabilities 604(n) by the weight of any one of the edges in the class neighborhood graph 504 that points away from the base class 702 and toward the class 506(n). For example, assume that the class 506(n) is the same as the base class 702. In this case, the high-weight directed edge 704 may point away from the base class 702 and toward the class 506(n). Therefore, the model component 318 may multiply the raw class probabilities 604(n) by the weight w high, resulting in the nth product. As another example, assume that class 506(n) is different from base class 702 and is known or expected to be a milling neighbor of base class 702. In this case, one of the one or more medium-weight directed edges 706 may point away from base class 702 and toward class 506(n). Therefore, model component 318 may multiply raw class probability 604(n) by weight w mid , resulting in the nth product. As yet another example, assume that class 506(n) is different from base class 702 and is known or expected not to be a milling neighbor of base class 702. In this case, one of the one or more low-weight directed edges 708 may point away from base class 702 and toward class 506(n). Therefore, model component 318 may multiply raw class probability 604(n) by weight w low , thus obtaining the nth product.

[0129] Such multiplication based on the weights of the high-weight directed edge 704, the one or more medium-weight directed edges 706, and the one or more low-weight directed edges 708 may result in a total of n products (e.g., one product per original class probability). In various aspects, the model component 318 may normalize such n products, such as by dividing each product by the sum of all n of those products. In various cases, such normalization may result in a plurality of modified class probabilities 710. In particular, the normalized version of the first product may be referred to as a modified class probability 710(1), which may be considered to be a modified or transformed version of the original class probability 604(1). In other words, the modified class probability 710(1) may be considered to indicate a modified likelihood that the slice 308 belongs to the class 506(1) (e.g., indicating a modified likelihood that the structure 305(1) is currently visible in or on the slice 308). Similarly, the normalized version of the nth product may be referred to as modified class probability 710(n), which may be considered a modified or transformed version of original class probability 604(n). That is, modified class probability 710(n) may be considered to indicate a modified likelihood that slice 308 belongs to class 506(n) (e.g., a modified likelihood that structure 305(n) is currently visible in or on slice 308).

[0130] In various cases, modified class probabilities 710(1)-710(n) may be considered collectively to be a plurality of modified class probabilities 710. In various aspects, classification label 404 may be considered to include or otherwise consist of a plurality of modified class probabilities 710.

[0131] To help clarify the various aspects described herein, consider Figures 8 to 9 Non-limiting examples shown.

[0132] Figure 8 A non-limiting example of a class neighborhood graph 504 is shown. In this non-limiting example, the plurality of classes 506 may include a total of five classes (e.g., n=5): class A representing structure A; class B representing structure B; class C representing structure C; class D representing structure D; and class E representing structure E. As shown, those five classes may be coupled in various ways via various edges. To avoid confusion due to excessive visual clutter, Figure 8 Only high-weight directed edges (e.g., 508) and medium-weight directed edges (e.g., 510) are depicted in this non-limiting example. Figure 8 Low-weight directed edges (e.g., 512) are visually omitted in order to prevent Figure 8 Overly confusing and incomprehensible. In this non-limiting example, w high =100,w mid =50, and w low =0.1.

[0133] As shown, in this non-limiting example, class A may have a high weight edge pointing to itself, a medium weight edge pointing to class B, and another medium weight edge pointing to class C. Although not explicitly shown, it should be understood that class A may therefore have a low weight edge pointing to class D and another low weight edge pointing to class E. Therefore, this can be interpreted to mean the following: class A is a milling neighbor of itself; class B is a milling neighbor of class A; class C is another milling neighbor of class A; class D is not a milling neighbor of class A; and class E is not a milling neighbor of class A.

[0134] As shown, in this non-limiting example, class B may have a high weight edge pointing to itself and a medium weight edge pointing to class C. Although not explicitly shown, it should be understood that class B may therefore have a low weight edge pointing to class D, another low weight edge pointing to class E, and yet another low weight edge pointing to class A. Therefore, this can be interpreted to mean the following: class B is a milling neighbor of itself; class C is a milling neighbor of class B; class D is not a milling neighbor of class B; class E is not a milling neighbor of class B; and class A is not a milling neighbor of class B.

[0135] As shown, in this non-limiting example, class C may have a high weight edge pointing to itself and a medium weight edge pointing to class D. Although not explicitly shown, it should be understood that class C may therefore have a low weight edge pointing to class E, another low weight edge pointing to class A, and yet another low weight edge pointing to class B. Therefore, this can be interpreted to mean the following: class C is a milling neighbor of itself; class D is a milling neighbor of class C; class E is not a milling neighbor of class C; class A is not a milling neighbor of class C; and class B is not a milling neighbor of class C.

[0136] As shown, in this non-limiting example, class D may have a high weight edge pointing to itself and a medium weight edge pointing to class E. Although not explicitly shown, it should be understood that class D may therefore have a low weight edge pointing to class A, another low weight edge pointing to class B, and yet another low weight edge pointing to class C. Therefore, this can be interpreted to mean the following: class D is a milling neighbor of itself; class E is a milling neighbor of class D; class A is not a milling neighbor of class D; class B is not a milling neighbor of class D; and class C is not a milling neighbor of class D.

[0137] As shown, in this non-limiting example, class E may have a high weight edge pointing to itself and a medium weight edge pointing to class A. Although not explicitly shown, it should be understood that class E may therefore have a low weight edge pointing to class B, another low weight edge pointing to class C, and yet another low weight edge pointing to class D. Therefore, this can be interpreted to mean the following: class E is a milling neighbor of itself; class A is a milling neighbor of class E; class B is not a milling neighbor of class E; class C is not a milling neighbor of class E; and class D is not a milling neighbor of class E.

[0138] Now, Fig. 9 A non-limiting example of raw classification labels 602 is shown. As shown, in this non-limiting example, the raw classification labels 602 may indicate a 20% raw probability for class A, a 5% raw probability for class B, a 30% raw probability for class C, a 40% raw probability for class D, and a 5% raw probability for class E. In various aspects, this non-limiting example of raw classification labels 602 may be based on Figure 8 The non-limiting example of the class neighborhood graph 504 shown may be modified.

[0139] For example, assume that the base class 702 (e.g., the previously predicted or inferred class) is class A. In this case, because Figure 8 The class neighborhood graph 504 shown has a 100 weight edge pointing from the base class 702 to class A, so the original class A probability of 20% can be multiplied by 100, resulting in a first product of 20 (e.g., 0.2*100=20). Similarly, because Figure 8 The class neighborhood graph 504 shown has a 50 weight edge pointing from the base class 702 to class B, so the original class B probability of 5% can be multiplied by 50, resulting in a second product of 2.5 (e.g., 0.05*50=2.5). Similarly, because Figure 8 The class neighborhood graph 504 shown has a 50 weight edge pointing from the base class 702 to class C, so the original class C probability of 30% can be multiplied by 50, resulting in a third product of 15 (e.g., 0.3*50=15). In a similar manner, because Figure 8The class neighborhood graph 504 shown has a 0.1 weight edge pointing from the base class 702 to class D, so the original class D probability of 40% can be multiplied by 0.1, resulting in a fourth product of 0.04 (e.g., 0.4*0.1=0.04). In addition, because Figure 8 The class neighborhood graph 504 shown has a 0.1 weight edge pointing from the base class 702 to class E, so the original class E probability of 5% can be multiplied by 0.1, resulting in a fifth product of 0.005 (e.g., 0.05*0.1=0.005). In various aspects, those five products can be normalized to yield a non-limiting example of a classification label 404. In practice, the sum of such five products can be 37.545 (e.g., 20+2.5+15+0.04+0.005=37.545), and thus the classification label 404 can be as follows: The modified class A probability can be 53% (e.g., ); the modified class B probability may be 7% (e.g., ); the modified class C probability may be 40% (e.g., 0.4); the modified class D probability may be 0% (e.g., ); and the modified class E probability may be 0% (e.g., ). In other words, in this case, it can be determined or inferred that structure A (e.g., base structure 702) was previously visible in or on the slice 308, the deep learning neural network 502 can predict that structure D is now currently visible in or on the slice 308 (e.g., the original class D probability is highest in the original classification label 602), and the prediction can be considered to be modified based on the modification of the class neighborhood graph 504, so that structure A is now inferred to still be currently visible in or on the slice 308 (e.g., the modified class A probability is highest in the classification label 404 of this example). In other words, the modification based on the class neighborhood graph 504 can be considered a type of robustness enhancement to the deep learning neural network 502. In fact, if structure A was visible in or on the slice 308 before the most recent milling round or iteration, structure D is now not visible because structure D is not a milled neighbor of structure A. Thus, the deep learning neural network 502 in this example may be considered to initially output a false positive, and the false positive may be considered to be corrected based on the modification of the class neighborhood graph 504. In other words, the modification based on the class neighborhood graph 504 as described herein may be considered to bias the deep learning neural network 502 toward predictions of milling neighbors and away from predictions of non-milling neighbors.

[0140] As another example, assume that the base class 702 (e.g., the previously predicted or inferred class) is class D instead of class A. In this case, because Figure 8The class neighborhood graph 504 shown has a 0.1 weight edge pointing from base class 702 to class A, so the original class A probability of 20% can be multiplied by 0.1, resulting in a first product of 0.02 (e.g., 0.2*0.1=0.02). Similarly, because Figure 8 The class neighborhood graph 504 shown has a 0.1 weight edge pointing from the base class 702 to class B, so the original class B probability of 5% can be multiplied by 0.1, resulting in a second product of 0.005 (e.g., 0.05*0.1=0.005). Similarly, because Figure 8 The class neighborhood graph 504 shown has a 0.1 weight edge pointing from the base class 702 to class C, so the original class C probability of 30% can be multiplied by 0.1, resulting in a third product of 0.03 (e.g., 0.3*0.1=0.03). In a similar manner, because Figure 8 The class neighborhood graph 504 shown has a 100 weight edge pointing from the base class 702 to class D, so the original class D probability of 40% can be multiplied by 100, resulting in a fourth product of 40 (e.g., 0.4*100=40). In addition, because Figure 8 The class neighborhood graph 504 shown has a 50 weight edge pointing from the base class 702 to class E, so the original class E probability of 5% can be multiplied by 50, resulting in a fifth product of 2.5 (e.g., 0.05*50=2.5). In various aspects, those five products can be normalized to yield a non-limiting example of a classification label 404. In practice, the sum of such five products can be 42.555 (e.g., 0.02+0.005+0.03+40+2.5=42.555), and thus the classification label 404 can be as follows: The modified class A probability can be 0% (e.g., ); the modified class B probability may be 0% (e.g., ); the modified class C probability may be 0% (e.g., ); the modified class D probability may be 94% (e.g., ); and the modified class E probability may be 6% (e.g., ). In other words, in this case, it can be determined or inferred that structure D (e.g., base structure 702) was previously visible in or on slice 308, the deep learning neural network 502 can predict (albeit with less confidence) that structure D is still currently visible in or on slice 308 (e.g., the original class D probability is highest in the original classification label 602), and the modification based on the class neighborhood graph 504 can be considered to modify the prediction so that structure D is inferred with very high confidence to still be currently visible in or on slice 308 (e.g., the modified class D probability is highest in the classification label 404 of this example). Therefore, not only can the modification based on the class neighborhood graph 504 be considered a type of robustness enhancement that biases the prediction of the deep learning neural network 502 toward milling neighbors and away from non-milling neighbors, but such modification can also be considered a type of preventive enhancement that biases the prediction to avoid transitioning from the previous class / structure to milling neighbors until the deep learning neural network 502 is strongly confident of such a transition.

[0141] In any case, the model component 318 can electronically generate the classification label 404 by executing the graph weighted neural network 402 on the image 306 (or on the device extracted from the image). In various aspects, the classification label 404 can be considered to indicate one of the multiple classes 506 and therefore one of the multiple structures 305. In particular, one of the multiple modified class probabilities 710 can be higher than the others. The highest or maximum modified class probability can correspond to a given one of the multiple classes 506, and therefore corresponds to a given one of the multiple structures 305. In other words, the classification label 404 can indicate that the slice 308 currently belongs to the given class. Equivalently, the classification label 404 can indicate that a given structure is currently visible in or on the slice 308.

[0142] Fig.10 A block diagram illustrating an example non-limiting system including target classes and milling instructions that can facilitate sheet endpoint positioning via a graph weighted neural network according to one or more embodiments described herein.

[0143] In various embodiments, the milling assembly 320 can electronically identify the target class 1002. In various aspects, the target class 1002 can be any one of the plurality of classes 506, and thus can represent any one of the plurality of structures 305 that is desired to be physically separated within the sheet 304. In various cases, the target class 1002 can be indicated or selected by any suitable user input. As a non-limiting example, a technician associated with the scientific instrument 302 can interact with any suitable human interface device (e.g., keyboard, keypad, touch screen, voice command system) of the scientific instrument 302 or the system 310 to manually select or specify which one of the plurality of classes 506 will be the target class 1002.

[0144] In various cases, the milling component 320 can electronically compare the target class 1002 to any of the classes indicated by the classification labels 404 .

[0145] If any of the classes indicated by the classification labels 404 are different from the target class 1002, the milling assembly 320 may electronically transmit milling instructions 1004 to the scientific instrument 302. In various aspects, the milling instructions 1004 may be considered to be any suitable electronic command that prompts or causes the scientific instrument 302 to perform an additional incremental milling round or iteration (e.g., prompts or causes the scientific instrument 302 to remove an additional 1 nm or 2 nm or less of material from the slice 308). In other words, if any of the classes indicated by the classification labels 404 are different from the target class 1002, the milling assembly 320 may conclude that any of the structures represented by the target class 1002 are not yet visible in or on the slice 308. Therefore, the milling assembly 320 may conclude that additional milling is required, and therefore the milling instructions 1004 are required. Note that any of the classes indicated by the classification labels 404 may be considered or viewed as being the base class 702 during the classification weighting procedure immediately following such additional milling round or iteration.

[0146] Conversely, if any of the classes indicated by the classification labels 404 are the same as the target class 1002, the milling component 320 may refrain from electronically transmitting the milling instructions 1004 to the scientific instrument 302. In fact, in this case, the milling component 320 may conclude that the end-point positioning of the sheet 304 is complete or final with respect to at least one side of the sheet 304. In some cases, if the other side or reverse side of the sheet 304 has been end-point positioned as described herein (e.g., has been milled so that any structure represented by the target class 1002 is visible in or on a cut surface of the reverse side), the milling component 320 may generate or transmit any suitable electronic message indicating that the end-point positioning of the sheet 304 is completely complete. On the other hand, if the other side or reverse side of the sheet 304 has not yet been endpoint positioned as described herein (e.g., has not yet been milled such that any structure represented by the target class 1002 is visible in or on a cut surface of that reverse side), the milling component 320 may generate or transmit any suitable electronic message indicating that endpoint positioning for the current side of the sheet 304 is complete and that endpoint positioning should next be performed on the reverse side of the sheet 304.

[0147] In any case, once endpoint positioning on sheet 304 is fully completed, any structures represented by target class 1002 may be considered physically separated within sheet 304 (e.g., the remaining portions of plurality of structures 305 may be removed from sheet 304 via milling). Accordingly, any downstream or subsequent analysis may then be performed on the separated structures (e.g., transmission electron microscopy analysis may be performed on the separated structures).

[0148] Figure 11 to Figure 12 Flowcharts illustrating example non-limiting computer-implemented methods 1100 and 1200 that can facilitate sheet endpoint localization via a graph weighted neural network according to one or more embodiments described herein. In various cases, the computer-implemented methods 1100 and 1200 can be facilitated by the system 310 .

[0149] First, consider Fig.11 In various embodiments, act 1102 may include accessing, by a device operatively coupled to a processor (e.g., 312) (e.g., via 316), a scientific instrument (e.g., 302) that may facilitate endpoint positioning of a sheet (e.g., 304). In various cases, a sheet may include a plurality of structures (e.g., 305).

[0150] In various aspects, act 1104 may include accessing, by a device (eg, via 316 or 320 ), user input indicating a target structure (eg, represented by 1002 ) among a plurality of structures.

[0151] In various cases, action 1106 may include accessing, by a device (e.g., via 316 or 318), a neighborhood graph (e.g., 504), the nodes (e.g., 506) of the neighborhood graph respectively representing multiple structures, and the weighted edges (e.g., 508, 510, 512) of the neighborhood graph respectively indicating which structures are milling neighbors of each other.

[0152] In various cases, action 1108 may include initializing, by the device (e.g., via 318), the infrastructure (e.g., 702). In some cases, any suitable initialization technique may be implemented. As a non-limiting example, the infrastructure may be automatically identified based on one or more raw classification labels generated by the deep learning classifier (e.g., if multiple consecutive raw classification labels are consistent with each other, any one of the structures that are consistent may be considered the first infrastructure). In various aspects, the computer-implemented method 1100 may then proceed to action 1202 of the computer-implemented method 1200.

[0153] Now, consider Fig.12In various embodiments, act 1202 may include commanding, by the device (eg, via any of 315 ), the scientific instrument to capture an image (eg, 306 ) of a section (eg, 308 ) of the slice.

[0154] In various aspects, action 1204 may include generating, by a device (e.g., via 318) and via executing a neural network (e.g., 502) on an image, a raw classification label (e.g., 602) indicating a probability that the slice contains a corresponding structure from among a plurality of structures (e.g., 604).

[0155] In various cases, action 1206 may include converting, by the device (e.g., via 318), the original classification label into a modified classification label (e.g., 404) by multiplying the probability of the original classification label by a corresponding weight (e.g., weights 704, 706, and 708) indicated by a neighborhood graph for the base structure.

[0156] In various cases, act 1208 may include determining, by the device (eg, via 320 ), whether the modified classification label indicates that the slice contains the target structure. If so, the computer-implemented method 1200 may end. If not, the computer-implemented method 1200 may proceed to act 1210 .

[0157] In various aspects, act 1210 may include commanding, by the device (eg, via 320 ), the scientific instrument to incrementally mill a slice of the slice.

[0158] In various cases, act 1212 may include redefining, by the device (eg, via 318 or 320), the base structure as any one of the plurality of structures indicated by the modified classification label. In various cases, computer-implemented method 1200 may then return to act 1202.

[0159] In order to make the original classification labels and modified classification labels described herein accurate, correct or reliable, the deep learning neural network 502 may first undergo training, such as Figures 13 to 15 described.

[0160] Fig.13 A block diagram of an example non-limiting system including training components and training data sets that can facilitate thin-sheet endpoint positioning via a graph weighted neural network according to one or more embodiments described herein is illustrated. As shown, in some cases, one or more software components 315 can include a training component 1302.

[0161] In various aspects, access component 316 can electronically receive, retrieve, or otherwise access training data set 1304 from any suitable source, and training component 1302 can electronically train deep learning neural network 502 on training data set 1304. Figure 14 to Figure 15 Non-limiting aspects are described.

[0162] Fig.14 An example non-limiting block diagram of a training data set 1304 is illustrated according to one or more embodiments described herein.

[0163] In various aspects, training data set 1304 may include a set of training images 1402. In various cases, for any suitable positive integer q, set of training images 1402 may include q images: training image 1402(1) through training image 1402(q). In various cases, each of set of training images 1402 may depict a respective slice that may be known to include the same structure (e.g., 305) as slice 304.

[0164] In various aspects, the training data set 1304 may include a set of true data classification labels 1404, which may respectively correspond to the set of training images 1402. Thus, since the set of training images 1402 may have q images, the set of true data classification labels 1404 may have q labels: true data classification label 1404(1) to true data classification label 1404(q). In various cases, each of the set of true data classification labels 1404 may be a correct or accurate classification label that is known or deemed to correspond to a corresponding one of the set of training images 1402. As a non-limiting example, the true data classification label 1404(1) may correspond to the training image 1402(1). Thus, the true data classification label 1404(1) may indicate a correct or accurate structure that is known or deemed to be visible in or on a slice depicted in the training image 1402(1). As another non-limiting example, the ground truth classification labels 1404 ( q ) may indicate correct or accurate structures that are known or believed to be visible in or on the slices depicted in the training image 1402 ( q ).

[0165] Fig.15 An example non-limiting block diagram showing how a deep learning neural network 502 may be trained on a training dataset 1304 is illustrated in accordance with one or more embodiments described herein.

[0166] In various aspects, prior to commencing training, the training component 1302 can initialize the trainable internal parameters (e.g., convolution kernels, weight matrices, bias values) of the deep learning neural network 502 in any suitable manner (e.g., via random initialization).

[0167] In various aspects, the training component 1302 may select a training image 1502 and a real data classification label 1504 corresponding to the training image 1502 from the training data set 1304. In various cases, the training component 1302 may execute the deep learning neural network 502 on the training image 1502 (or on the device line extracted from the training image), so that the deep learning neural network 502 produces an output 1506. More specifically, in some cases, the training component 1302 may feed the training image 1502 (or the device line extracted from the training image) to an input layer of the deep learning neural network 502, the training image 1502 (or the device line extracted from the training image) may complete a forward pass through one or more hidden layers of the deep learning neural network 502, and the output layer of the deep learning neural network 502 may operate on the output 1506 based on the activation map or feature map provided by the one or more hidden layers of the deep learning neural network 502.

[0168] Note that the format, size, or dimensions of output 1506 may be dictated by the number, arrangement, size, or other characteristics of the neurons, convolution kernels, or other internal parameters of the output layer (or any other layer) of deep learning neural network 502. Thus, output 1506 may be forced to have any desired format, size, or dimension by adding, removing, or otherwise adjusting characteristics of the output layer (or any other layer) of deep learning neural network 502. Thus, output 1506 may be considered to be the predicted or inferred classification label that deep learning neural network 502 believes should correspond to training image 1502. In contrast, true data classification label 1504 may be considered to be the correct or accurate classification label that is known or believed to correspond to training image 1502. Note that if deep learning neural network 502 has not been trained or has been trained very little to date, output 1506 may be very inaccurate (e.g., may be very different from true data classification label 1504).

[0169] In various aspects, the training component 1302 can compute any suitable error or loss (e.g., mean absolute error, mean squared error, cross entropy error) between the output 1506 and the true data classification label 1504. In various cases, the training component 1302 can incrementally update the trainable internal parameters of the deep learning neural network 502 via back propagation (e.g., stochastic gradient descent) driven by the computed error or loss.

[0170] In various cases, this execution and updating procedure can be repeated for any suitable number of training images (e.g., for each training image in training dataset 1304). This can ultimately cause the trainable internal parameters of deep learning neural network 502 to become iteratively optimized for accurately classifying visible structures in input images of slice sections (at least slices whose engineering design matches the engineering design of slice 304). In various aspects, training component 1302 can implement any suitable training batch size, any suitable error, loss, or objective function, or any suitable training termination criteria.

[0171] Although the above description mainly describes the deep learning neural network 502 as being trained in a supervised manner, this is only a non-limiting example for ease of illustration and explanation. In various cases, the training component 1302 can implement any other suitable training paradigm (e.g., unsupervised training, reinforcement learning) to train the deep learning neural network 502.

[0172] In any case, because the class neighborhood graph 504 can be utilized as described herein to improve or enhance the classification accuracy of the deep learning neural network 502, the deep learning neural network 502 does not need to undergo extensive or large amounts of training. In other words, the training data set 1304 does not need to include thousands of images, the curation of which would require sacrificing or destroying tens or hundreds of slices. Instead, the training data set 1304 may include far fewer training images (e.g., on the order of tens of images), which may require sacrificing or destroying far fewer slices, and the deep learning neural network 502 may still effectively achieve satisfactory classification accuracy due to the implementation of the class neighborhood graph 504. In fact, the inventors have made simplifications to practice the embodiments described herein, in which satisfactory classification accuracy is achieved for the deep learning neural network 502, even though the deep learning neural network 502 is trained only on images captured from only six sacrificial slices (e.g., tens of training images total).

[0173] Figures 16 to 20 Example non-limiting experimental results according to one or more embodiments described herein are illustrated.

[0174] Fig.16 An image 1600 showing a real-world slice of a real-world slice is illustrated.

[0175] Fig.17 Illustrated is device lines 1700 extracted or cropped from image 1600 via the sliding window technique mentioned above.

[0176] Fig.18A class neighborhood graph 1800 is shown that is utilized in a non-limiting example embodiment that is simplified for practice by the inventors. In particular, the implementation is simplified for practice involving a wafer containing a FinFET. As shown by the class neighborhood graph 1800, the FinFET is broken down into four different classes: an incoming source-drain class; a source-drain class; an incoming gate class; and a gate class. As shown, the incoming source-drain class is assigned a high weight edge to itself, a medium weight edge to the source-drain class, and a low weight edge to both the incoming gate class and the gate class (as described above, for ease of viewing, the edges are shown in the figure). Fig.18 (The low weight edges are omitted in the figure to avoid visual clutter). In addition, as shown in the figure, the Source Drain class is assigned a high weight edge with itself, a medium weight edge with the Gate class, and a low weight edge with both the Gate class and the Source Drain class. In addition, the Gate class is assigned a high weight edge with itself, a medium weight edge with the Gate class, and a low weight edge with both the Source Drain class and the Source Drain class. Finally, the Gate class is assigned a high weight edge with itself, a medium weight edge with the Source Drain class, and a low weight edge with both the Gate class and the Source Drain class.

[0177] Fig.19 Depicted is a graph 1900 showing raw probabilities generated by the deep learning neural network 502 after training for a sequence of slice images of a particular slice containing a FinFET. An example of graph 1900 is as follows: number 1902 represents the gate class; number 1904 represents entry into the source-drain class; number 1906 represents the source-drain class; and number 1908 represents entry into the gate class. As shown, near the interface between different classes / structures, the raw probabilities become very noisy and uncertain. In other words, it can be considered that the deep learning neural network 502 becomes significantly uncertain or unreliable near structure / class transitions (e.g., for slices near the location where one structure ends and another structure begins). In yet other words, it can be considered that the deep learning neural network 502 may misclassify slices near structure / class transitions.

[0178] Fig. 20 Depicted is a diagram showing a class neighborhood graph 1800 as described herein by Fig.192000 of modified probabilities achieved by multiplying and normalizing the original probabilities shown. The example of graph 2000 is as follows: number 2002 represents the gate class; number 2004 represents the entry source drain class; number 2006 represents the source drain class; and number 2008 represents the entry gate class. As shown, the modified probabilities are much smoother and less noisy than the original probabilities, especially at or near the structure / class transition. Therefore, it can be considered that the modified probabilities are less vulnerable to misclassification or influence than the original probabilities. In other words, the various embodiments described herein can be considered to be clever or creative techniques for helping to prevent or undo false positives or false negatives generated by the deep learning neural network 502. In other words, the various embodiments described herein can be considered to be creative techniques for reducing hesitation, noise or uncertainty of the deep learning neural network 502 for thin slice sections located at or near the structure / class transition or interface.

[0179] It is to be understood and appreciated that all specific numerical values ​​presented herein are merely non-limiting examples.

[0180] The scientific instrument systems, methods, or techniques disclosed herein may include (e.g., Fig.23 The user local computing device 2320 discussed above) interacts with a human user. These interactions may include providing information to the user (e.g., about scientific instruments (such as Fig.23 information about the operation of a scientific instrument 2310), information about a sample being analyzed or other test or measurement performed by the scientific instrument, information retrieved from a local or remote database, or other information) or providing the user with the option of inputting commands (e.g., controlling a scientific instrument such as Fig.23 In some embodiments, these interactions may be performed via a graphical user interface (GUI) that includes a display device (e.g., a graphical user interface such as a display device such as a display device such as a computer program product such as a computer program product, or a display device such as a computer program product, such as a graphics card or a graphics card). Fig. 22 2210) that provides output to a user and / or prompts the user (e.g., via the reference 2210 herein). Fig. 22 Input may be provided by one or more input devices, such as a keyboard, mouse, trackpad, or touch screen, included in other I / O devices 2212 discussed herein. The scientific instrument systems, methods, or techniques disclosed herein may include any GUI suitable for interacting with a user.

[0181] Fig.21An example graphical user interface 2100 (hereinafter referred to as "GUI 2100") is depicted that can be used to perform some or all of the supporting methods or techniques disclosed herein according to various embodiments. In various aspects, GUI 2100 can be provided in a scientific instrument support system (e.g., as described herein with reference to Fig.23 A computing device (e.g., a scientific instrument support system 2300 as discussed herein) Fig. 22 Any suitable electronic display (e.g., the computing device 2200 discussed herein) Fig. 22 The display device 2210 discussed above) and the user or technician may use any suitable input device (e.g., Fig. 22 The GUI 2100 may be interacted with by any of the other I / O devices 2212 discussed and input technologies (e.g., cursor movement, motion capture, facial recognition, gesture detection, voice recognition, button activation).

[0182] GUI 2100 may include a data display area 2102 , a data analysis area 2104 , a scientific instrument control area 2106 , and a settings area 2108 . Fig.21 The specific number and arrangement of regions depicted are merely exemplary, and any number and arrangement of regions (including any desired features) may be included in other embodiments of GUI 2100.

[0183] Data display area 2102 may display data generated by a scientific instrument (e.g., Fig.23 Data generated by the scientific instrument 2310) in question.

[0184] Data analysis area 2104 can display any suitable data analysis results (e.g., the results of analyzing the data illustrated in data display area 2102 or other data). In some embodiments, data display area 2102 and data analysis area 2104 can be combined in GUI 2100 (e.g., including data output from a scientific instrument and some analysis of the data in a common graph or area).

[0185] The scientific instrument control area 2106 may include options that allow a user or technician to control the scientific instrument (e.g., Fig.23 For example, the scientific instrument control area 2106 may include configurable parameters for controlling the operation of such a scientific instrument (e.g., configurable parameters for controlling voltage or current of the scientific instrument, configurable parameters for controlling the internal temperature of the scientific instrument, or configurable parameters for controlling the flow rate of a fluid of the scientific instrument).

[0186] The settings area 2108 may include options that allow the user or technician to control any features or functions of the GUI 2100 (or other GUIs) or to perform common computational operations with respect to the data display area 2102 and the data analysis area 2104 (e.g., saving data to a storage device (such as a computer program product described herein)). Fig. 22 The storage device 2204 in question, sending the data to another user, marking the data).

[0187] As described above, the scientific instrument module 102 may be implemented by one or more computing devices. Fig. 22 2200 that can perform some or all of the scientific instrument support methods or techniques disclosed herein according to various embodiments. In some embodiments, the scientific instrument module 102 can be implemented by a single instance of the computing device 2200 or multiple instances of the computing device 2200. In addition, as discussed below, the computing device 2200 (or multiple instances of the computing device) that implements the scientific instrument module 102 can be Fig.23 A portion of one or more of scientific instrument 2310, user local computing device 2320, service local computing device 2330, or remote computing device 2340.

[0188] The computing device 2200 is illustrated as having multiple components, but any one or more of these components may be omitted or duplicated depending on the application and settings. In some embodiments, some or all of the components included in the computing device 2200 may be attached to one or more motherboards and encapsulated in a housing (e.g., comprising plastic, metal, or other materials). In some embodiments, some of these components may be constructed onto a single system on a chip (SoC) (e.g., the SoC may include one or more instances of the processing device 2202 and one or more instances of the storage device 2204). Additionally, in various embodiments, the computing device 2200 may omit the processor 2202 and the storage device 2204. Fig. 22 One or more of the components illustrated may include interface circuitry (not shown) for coupling to one or more omitted components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI) interface, a controller area network (CAN) interface, a serial peripheral interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, the computing device 2200 may omit the display device 2210, but may include display device interface circuitry (e.g., a connector and driver circuitry) to which the display device 2210 may be coupled.

[0189] The computing device 2200 may include a processing device 2202 (e.g., one or more processing devices). As used herein, the term "processing device" may refer to any device or portion of a device that processes electronic data from a register or memory to convert the electronic data into other electronic data that can be stored in the register or memory. The processing device 2202 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (specialized processors that execute cryptographic algorithms in hardware), server processors, or any other suitable processing devices.

[0190] The computing device 2200 may include a storage device 2204 (e.g., one or more storage devices). The storage device 2204 may include one or more memory devices, such as a random access memory (RAM) (e.g., a static RAM (SRAM) device, a magnetic RAM (MRAM) device, a dynamic RAM (DRAM) device, a resistive RAM (RRAM) device, or a conductive bridge RAM (CBRAM) device), a hard drive-based memory device, a solid-state memory device, a networked drive, a cloud drive, or any combination of memory devices. In some embodiments, the storage device 2204 may include a memory that shares a die with the processing device 2202. In such embodiments, the memory may be used as a cache memory and may include, for example, an embedded dynamic random access memory (eDRAM) or a spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, the storage device 2204 may include a non-transitory computer-readable medium having instructions thereon that, when executed by one or more processing devices (e.g., the processing device 2202), cause the computing device 2200 to perform any appropriate method or portion of the methods disclosed herein.

[0191] The computing device 2200 may include an interface device 2206 (e.g., one or more instances of the interface device 2206). The interface device 2206 may include one or more communication chips, connectors, or other hardware and software to manage communications between the computing device 2200 and other computing devices. For example, the interface device 2206 may include circuits for managing wireless communications for transmitting data to and from the computing device 2200. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, and communication channels that can transmit data through a non-solid medium using modulated electromagnetic radiation. The term does not imply that the associated device does not include any wires, although in some embodiments it may not include any wires. The circuitry included in the interface device 2206 for managing wireless communications may implement any of a variety of wireless standards or protocols, including, but not limited to, Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.11 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendments), Long Term Evolution (LTE) projects and any amendments, updates and / or revisions (e.g., Advanced LTE projects, Ultra Mobile Broadband (UMB) projects (also known as "3GPP2"). In some embodiments, the circuitry included in the interface device 2206 for managing wireless communications may operate in accordance with a Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, the circuitry included in the interface device 2206 for managing wireless communications may operate in accordance with Enhanced Data GSM Evolution (EDGE), GSM The interface device 2206 may be configured to operate in accordance with an EDGE Radio Access Network (GERAN), a Universal Terrestrial Radio Access Network (UTRAN), or an Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in the interface device 2206 for managing wireless communications may operate in accordance with code division multiple access (CDMA), time division multiple access (TDMA), digital enhanced cordless telecommunications (DECT), evolution data optimized (EV-DO) and its derivatives, and any other wireless protocols designated as 3G, 4G, 5G and higher. In some embodiments, the interface device 2206 may include one or more antennas (e.g., one or more antenna arrays) to receive and / or transmit wireless communications.

[0192] In some embodiments, interface device 2206 may include a circuit for managing wired communication, such as electrical communication protocol, optical communication protocol or any other suitable communication protocol. For example, interface device 2206 may include a circuit supporting communication according to Ethernet technology. In some embodiments, interface device 2206 may support both wireless and wired communication, or may support multiple wired communication protocols or multiple wireless communication protocols. For example, a first group of circuits of interface device 2206 may be dedicated to shorter range wireless communication such as Wi-Fi or Bluetooth, and a second group of circuits of interface device 2206 may be dedicated to longer range wireless communication such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, a first group of circuits of interface device 2206 may be dedicated to wireless communication, and a second group of circuits of interface device 2206 may be dedicated to wired communication.

[0193] The computing device 2200 may include a battery / power circuit 2208. The battery / power circuit 2208 may include one or more energy storage devices (e.g., batteries or capacitors) or circuits for coupling components of the computing device 2200 to an energy source separate from the computing device 2200 (e.g., AC line power).

[0194] Computing device 2200 may include display device 2210 (eg, multiple display devices). Display device 2210 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light emitting diode display, or a flat panel display.

[0195] The computing device 2200 may include other input / output (I / O) devices 2212. For example, the other I / O devices 2212 may include one or more audio output devices (e.g., speakers, headphones, earbuds, alarms), one or more audio input devices (e.g., microphones or microphone arrays), a positioning device (e.g., a GPS device that communicates with a satellite-based system to receive the location of the computing device 2200), an audio codec, a video codec, a printer, a sensor (e.g., a thermocouple or other temperature sensor, a humidity sensor, a pressure sensor, a vibration sensor, an accelerometer, a gyroscope), an image capture device such as a camera, a keyboard, a cursor control device such as a mouse, a stylus, a trackball, or a touchpad, a barcode reader, a quick response (QR) code reader, or a radio frequency identification (RFID) reader.

[0196] The computing device 2200 may have any suitable form factor suitable for its applications and settings, such as a handheld or mobile computing device (e.g., a cell phone, a smart phone, a mobile Internet device, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra-mobile personal computer), a desktop computing device, or a server computing device or other networked computing component.

[0197] One or more computing devices implementing any scientific instrument module, method, or technique disclosed herein may be part of a scientific instrument support system. Fig.23 23 is a block diagram of an example scientific instrument support system 2300 in which some or all of the scientific instrument support methods disclosed herein may be performed, according to various embodiments. The scientific instrument modules, methods, or techniques disclosed herein (e.g., scientific instrument module 102, computer-implemented method 200, system 310) may be implemented by one or more of a scientific instrument 2310, a user local computing device 2320, a service local computing device 2330, or a remote computing device 2340 of the scientific instrument support system 2300.

[0198] Any of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may include any implementation of the computing device 2200, and any of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the form of any appropriate implementation of the implementation of the computing device 2200.

[0199] The scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may each include a processing device 2302, a storage device 2304, and an interface device 2306. The processing device 2302 may take any suitable form, including any form of the processing device 2202, and the processing devices 2302 included in different devices of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the same form or different forms. The storage device 2304 may take any suitable form, including any form of the storage device 2204, and the storage devices 2304 included in different devices of the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the same form or different forms. The interface device 2306 may take any suitable form, including any form of the interface device 2206, and the interface device 2306 included in different devices in the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, or the remote computing device 2340 may take the same form or different forms.

[0200] Scientific instrument 2310, user local computing device 2320, service local computing device 2330, and remote computing device 2340 may communicate with other elements of scientific instrument support system 2300 via communication path 2308. Communication path 2308 may communicatively couple interface device 2306 of different ones of the elements of scientific instrument support system 2300, as shown, and may be a wired or wireless communication path (e.g., in accordance with any of the communication technologies discussed herein with reference to interface device 2206). Fig.23 The particular scientific instrument support system 2300 depicted includes communication paths between each pair of devices in the scientific instrument 2310, the user local computing device 2320, the service local computing device 2330, and the remote computing device 2340, but this "fully connected" specific implementation is merely illustrative, and in various embodiments, various communication paths in the communication paths 2308 may not be present. For example, in some embodiments, the service local computing device 2330 may lack a direct communication path 2308 between its interface device 2306 and the interface device 2306 of the scientific instrument 2310, but may instead communicate with the scientific instrument 2310 via the communication path 2308 between the service local computing device 2330 and the user local computing device 2320, as well as the communication path 2308 between the user local computing device 2320 and the scientific instrument 2310.

[0201] Scientific instrument 2310 may include any suitable scientific instrument, such as scientific instrument 302 .

[0202] The user-local computing device 2320 can be a computing device local to a user of the scientific instrument 2310 (e.g., according to any of the embodiments of the computing device 2200). In some embodiments, the user-local computing device 2320 can also be local to the scientific instrument 2310, but this is not necessarily the case; for example, a user-local computing device 2320 in a user's home or office can be remote from the scientific instrument 2310, but in communication with the scientific instrument so that the user can use the user-local computing device 2320 to control or access data from the scientific instrument 2310. In some embodiments, the user-local computing device 2320 can be a laptop, a smart phone, or a tablet device. In some embodiments, the user-local computing device 2320 can be a portable computing device.

[0203] The service local computing device 2330 can be a computing device local to the entity serving the scientific instrument 2310 (e.g., according to any of the embodiments of the computing device 2200). For example, the service local computing device 2330 can be a device local to the manufacturer of the scientific instrument 2310 or to a third-party service company. In some embodiments, the service local computing device 2330 can communicate (e.g., via a direct communication path 2308 or via multiple "indirect" communication paths 2308, as discussed above) with the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 to receive data regarding the operation of the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., self-test results of the scientific instrument 2310, calibration coefficients used by the scientific instrument 2310, measurements of sensors associated with the scientific instrument 2310). In some embodiments, the service local computing device 2330 may communicate with the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., via a direct communication path 2308 or via multiple "indirect" communication paths 2308, as discussed above) to transfer data to the scientific instrument 2310, the user local computing device 2320, or the remote computing device 2340 (e.g., to update programming instructions (such as firmware) in the scientific instrument 2310, to initiate execution of a test or calibration sequence in the scientific instrument 2310, to update programming instructions (such as software) in the user local computing device 2320 or the remote computing device 2340). A user of the scientific instrument 2310 may utilize the scientific instrument 2310 or the user local computing device 2320 to communicate with the service local computing device 2330 to report problems with the scientific instrument 2310 or the user local computing device 2320, to request a visit from a technician to improve the operation of the scientific instrument 2310, to order consumables or replacement parts associated with the scientific instrument 2310, or for other purposes.

[0204] Remote computing device 2340 can be a computing device remote from scientific instrument 2310 or user local computing device 2320 (e.g., according to any of the embodiments of computing device 2200 discussed herein). In some embodiments, remote computing device 2340 can be included in a data center or other large-scale server environment. In some embodiments, remote computing device 2340 can include a network attached storage device (e.g., as part of storage device 2304). Remote computing device 2340 can store data generated by scientific instrument 2310, perform analysis of data generated by scientific instrument 2310 (e.g., according to programming instructions), facilitate communication between user local computing device 2320 and scientific instrument 2310, or facilitate communication between service local computing device 2330 and scientific instrument 2310.

[0205] In some embodiments, the Fig.23 One or more of the illustrated elements of the scientific instrument support system 2300. In addition, in some embodiments, there may be Fig.23 The scientific instrument support system 2300 may include multiple of the various elements of the elements of the scientific instrument support system 2300. For example, the scientific instrument support system 2300 may include multiple user-local computing devices 2320 (e.g., different user-local computing devices 2320 associated with different users or located at different locations). In another example, the scientific instrument support system 2300 may include multiple scientific instruments 2310, all of which are in communication with the service local computing device 2330 and / or the remote computing device 2340; in such an embodiment, the service local computing device 2330 may monitor these multiple scientific instruments 2310, and the service local computing device 2330 may cause updates or other information to be "broadcasted" to the multiple scientific instruments 2310 simultaneously. The different scientific instruments 2310 in the scientific instrument support system 2300 may be close to each other (e.g., in the same room) or far away from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, the scientific instrument 2310 may be connected to an Internet of Things (IoT) stack that allows the scientific instrument 2310 to be directed and controlled through a web-based application, a virtual or augmented reality application, a mobile application, or a desktop application. Any of these applications may be accessed by a user operating a user-local computing device 2320 that communicates with the scientific instrument 2310 through an intermediary remote computing device 2340. In some embodiments, the scientific instrument 2310 may be sold by a manufacturer along with one or more associated user-local computing devices 2320 as part of a local scientific instrument computing unit 2312.

[0206] In some embodiments, the different scientific instruments 2310 included in the scientific instrument support system 2300 may be different types of scientific instruments 2310; for example, one scientific instrument 2310 may be a mass spectrometer, while another scientific instrument 2310 may be a chromatograph or an autosampler. In some such embodiments, the remote computing device 2340 or the user's local computing device 2320 may combine data from different types of scientific instruments 2310 included in the scientific instrument support system 2300.

[0207] In various cases, the machine learning algorithm or model may be implemented in any suitable manner to facilitate any suitable aspect described herein. In order to facilitate some of the above machine learning aspects of various embodiments, consider the following discussion of artificial intelligence (AI). Various embodiments described herein may employ artificial intelligence to facilitate automation of one or more features or functions. These components may employ various AI-based schemes to perform various embodiments / examples disclosed herein. In order to provide or assist in the numerous determinations described herein (e.g., determination, ascertainment, inference, calculation, prediction, prognosis, estimation, derivation, forecast, detection, calculation), the components described herein may examine all or a subset of the data to which they are granted access rights, and may provide a set of observations such as those captured via events or data to infer or determine the state of a system or environment. For example, determination may be employed to identify a specific context or action, or a probability distribution of a state may be generated. These determinations may be probabilistic; that is, a probability distribution of a state of interest is calculated based on consideration of data and events. Determination may also refer to the techniques employed to compose a higher-level event from a set of events or data.

[0208] This determination can result in constructing new events or actions from a set of observed events or stored event data, regardless of whether the events are closely related in time and whether the events and data come from one or several event and data sources. The components disclosed herein can employ various classification (explicit training (e.g., via training data) and implicit training (e.g., via observed behavior, preferences, historical information, receiving external information, etc.)) schemes or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) in connection with performing automatic or determined actions related to the claimed subject matter. Thus, the classification scheme or system can be used to automatically learn and perform multiple functions, actions, or determinations.

[0209] The classifier can take the input attribute vector z=(z1,z2,z3,z4,z n) is mapped to the confidence that the input belongs to a certain class, such as f(z) = confidence(class). This classification can use probabilistic or statistical-based analysis (for example, taking into account the utility and cost of the analysis) to determine the actions to be automatically performed. Support vector machine (SVM) can be an example of a classifier that can be used. SVM operates by finding a hypersurface in the space of possible inputs, where the hypersurface attempts to separate triggering criteria from non-triggering events. Intuitively, this makes the classification correct for test data that is close to but not the same as the training data. Other directed and non-directed model classification methods include, for example, naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that provide different independent patterns, any of which can be used. The classification used in this article also includes statistical regression for developing priority models.

[0210] To provide additional context for the various embodiments described herein, Fig.24 The following discussion is intended to provide a brief, general description of a suitable computing environment 2400 in which various embodiments of the embodiments described herein may be implemented. Although the embodiments have been described above in the general context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in conjunction with other program modules or as a combination of hardware and software.

[0211] Generally, program modules include routines, programs, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In addition, those skilled in the art will appreciate that the methods of the present invention can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, and personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc., each of which can be operably coupled to one or more associated devices.

[0212] The embodiments shown herein can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0213] Computing devices typically include various media, which may include computer-readable storage media, machine-readable storage media, or communication media, the two terms being used differently in this article as shown below. Computer-readable storage media or machine-readable storage media can be any available storage media that can be accessed by a computer, and include volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable storage media or machine-readable storage media can be implemented in conjunction with any method or technology for storing information (e.g., computer-readable or machine-readable instructions, program modules, structured data, or unstructured data).

[0214] Computer-readable storage media may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD ROM), digital versatile disc (DVD), Blu-ray disc (BD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, solid-state drives or other solid-state storage devices, or other tangible or non-transitory media that can be used to store the desired information. In this regard, the terms "tangible" or "non-transitory" as applied to storage, memory, or computer-readable media herein should be understood to exclude only the propagation of transient signals themselves as a modifier, and do not disclaim all rights to standard storage, memory, or computer-readable media that are not merely propagation of transient signals themselves.

[0215] Computer-readable storage media may be accessed by one or more local or remote computing devices, such as via access requests, queries, or other data retrieval protocols, for various operations regarding the information stored by the media.

[0216] Communication media typically includes computer readable instructions, data structures, program modules, or other structured or unstructured data in a data signal (such as a modulated data signal, such as a carrier wave or other transport mechanism), and includes any information delivery or transmission media. The term "modulated data signal" or signal refers to a signal whose one or more characteristics are set or changed so as to encode information in one or more signals. By way of example, but not limitation, communication media include wired media (such as a wired network or direct-wired connection) and wireless media (such as acoustic, RF, infrared and other wireless media).

[0217] Reference again Fig.24, an example environment 2400 for implementing various embodiments of various aspects described herein includes a computer 2402, which includes a processing unit 2404, a system memory 2406, and a system bus 2408. The system bus 2408 couples system components including, but not limited to, the system memory 2406 to the processing unit 2404. The processing unit 2404 can be any of a variety of commercially available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 2404.

[0218] The system bus 2408 may be any of several types of bus structures and may further interconnect with a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 2406 includes ROM 2410 and RAM 2412. A basic input / output system (BIOS) may be stored in a nonvolatile memory such as ROM, erasable programmable read-only memory (EPROM), EEPROM, wherein the BIOS contains the basic routines that help transfer information between elements within the computer 2402, such as during startup. The RAM 2412 may also include high-speed RAM, such as static RAM for caching data.

[0219] The computer 2402 also includes an internal hard disk drive (HDD) 2414 (e.g., EIDE, SATA), one or more external storage devices 2416 (e.g., magnetic floppy disk drive (FDD) 2416, memory stick or flash drive reader, memory card reader, etc.), and a drive 2420, such as a solid state drive, an optical drive, which can read or write from a disk 2422 (such as a CD-ROM disk, DVD, BD, etc.). Alternatively, in the case of a solid state drive, the disk 2422 will not be included unless it is separate. Although the internal HDD 2414 is illustrated as being located within the computer 2402, the internal HDD 2414 can also be configured for external use in a suitable chassis (not illustrated). In addition, although not shown in the environment 2400, a solid state drive (SSD) can be used to supplement or replace the HDD 2414. The HDD 2414, external storage device 2416, and drive 2420 may be connected to the system bus 2408 via a HDD interface 2424, an external storage interface 2426, and a drive interface 2428, respectively. The interface 2424 for an external drive specific implementation may include at least one or both of a Universal Serial Bus (USB) and an Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technology. Other external drive connection technologies are also within the contemplated scope of the embodiments described herein.

[0220] The drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, etc. For the computer 2402, the drives and storage media accommodate the storage of any data in a suitable digital format. Although the above description of computer-readable storage media refers to the corresponding type of storage device, it should be understood by those skilled in the art that other types of storage media (whether currently existing or developed in the future) that can be read by a computer can also be used in the example operating environment, and further, any such storage media can contain computer-executable instructions for performing the methods described herein.

[0221] A number of program modules may be stored in the drives and RAM 2412, including an operating system 2430, one or more applications 2432, other program modules 2434, and program data 2436. All or portions of the operating system, applications, modules, or data may also be cached in the RAM 2412. The systems and methods described herein may be implemented using various commercially available operating systems or combinations of operating systems.

[0222] Computer 2402 may optionally include emulation technology. For example, a virtual machine hypervisor (not shown) or other intermediary may emulate the hardware environment for operating system 2430, and the emulated hardware may optionally be different from the hardware environment of the operating system 2430. Fig.24 The illustrated hardware. In this embodiment, the operating system 2430 may include one of the multiple virtual machines (VMs) hosted at the computer 2402. In addition, the operating system 2430 may provide a runtime environment, such as a Java runtime environment or a .NET framework, for the application 2432. The runtime environment is a consistent execution environment that allows the application 2432 to run on any operating system that includes a runtime environment. Similarly, the operating system 2430 may support containers, and the application 2432 may be in the form of containers, which are lightweight, independent, executable software packages that include, for example, the application's code, runtime, system tools, system libraries, and settings.

[0223] In addition, the computer 2402 may be equipped with a security module, such as a trusted processing module (TPM). For example, using a TPM, the boot component hashes the next boot component in time and waits for the result to match the security value before loading the next boot component. This process can be performed at any layer in the code execution stack of the computer 2402, for example, applied to the application execution level or the operating system (OS) kernel level, thereby achieving security of code execution at any level.

[0224] A user may enter commands and information into the computer 2402 through one or more wired / wireless input devices (e.g., a keyboard 2438, a touch screen 2440, and a pointing device such as a mouse 2442). Other input devices (not shown) may include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control or other remote control, a joystick, a virtual reality controller or virtual reality headset, a game controller, a stylus, an image input device (e.g., a camera), a gesture sensor input device, a visual movement sensor input device, an emotion or facial detection device, a biometric input device (e.g., a fingerprint or iris scanner), and the like. These and other input devices are typically connected to the processing unit 2404 through an input device interface 2444, which may be coupled to the system bus 2408, but may also be connected through other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR port, a 3D image input device, or a fingerprint or iris scanner. Interfaces, etc.

[0225] A monitor 2446 or other type of display device may also be connected to the system bus 2408 via an interface, such as a video adapter 2448. In addition to the monitor 2446, computers typically include other peripheral output devices (not shown), such as speakers, printers, and the like.

[0226] The computer 2402 may operate in a networking environment using logical connections to one or more remote computers, such as remote computer 2450, via wired or wireless communications. The remote computer 2450 may be a workstation, server computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer device, or other common network node, and typically includes many or all of the elements described relative to the computer 2402, but only the memory / storage device 2452 is illustrated for simplicity. The depicted logical connections include wired / wireless connections to a local area network (LAN) 2454 or a larger network, such as a wide area network (WAN) 2456. Such LAN and WAN networking environments are common in offices and companies and facilitate the establishment of enterprise-wide computer networks, such as intranets, all of which can be connected to a global communication network, such as the Internet.

[0227] When used in a LAN networking environment, the computer 2402 may be connected to a local network 2454 through a wired or wireless communication network interface or adapter 2458. The adapter 2458 may facilitate wired or wireless communication with the LAN 2454, which may also include a wireless access point (AP) disposed thereon for communicating with the adapter 2458 in a wireless mode.

[0228] When used in a WAN networking environment, the computer 2402 may include a modem 2460 or may be connected to a communication server on the WAN 2456 via other means for establishing communications over the WAN 2456 (such as over the Internet). The modem 2460 may be connected to the system bus 2408 via the input device interface 2444, and may be an internal or external device, and may be a wired or wireless device. In a networking environment, program modules depicted relative to the computer 2402 or portions thereof may be stored in the remote memory / storage device 2452. It should be appreciated that the network connections shown are exemplary, and other means of establishing a communications link between the computers may be used.

[0229] When used in a LAN or WAN networking environment, the computer 2402 may access a cloud storage system or other network-based storage system in addition to or as an alternative to the external storage device 2416 as described above, such as, but not limited to, a network virtual machine that provides one or more aspects of storage or processing of information. Typically, the connection between the computer 2402 and the cloud storage system may be established through the LAN 2454 or WAN 2456, for example, by an adapter 2458 or a modem 2460, respectively. When the computer 2402 is connected to the associated cloud storage system, the external storage interface 2426 may manage the storage provided by the cloud storage system with the help of the adapter 2458 or the modem 2460, as it manages other types of external storage. For example, the external storage interface 2426 may be configured to provide access to cloud storage sources as if those sources were physically connected to the computer 2402.

[0230] Computer 2402 is operable to communicate with any wireless device or entity that operates in a wireless communication manner (e.g., printers, scanners, desktop or portable computers, portable data assistants, communication satellites, any equipment or location associated with a wirelessly detectable tag (e.g., kiosks, newsstands, store shelves, etc.), and telephones). This may include Wireless Fidelity (Wi-Fi) and Wireless technology. Therefore, the communication can be a predefined structure like traditional networks or an ad hoc communication between at least two devices.

[0231] Fig.2525 is a schematic block diagram of an example computing environment 2500 with which the disclosed subject matter can interact. The example computing environment 2500 includes one or more clients 2510. The client 2510 can be hardware or software (e.g., a thread, a process, a computing device). The example computing environment 2500 also includes one or more servers 2530. The server 2530 can also be hardware or software (e.g., a thread, a process, a computing device). For example, the server 2530 can accommodate threads to perform conversions by adopting one or more embodiments described herein. One possible communication between the client 2510 and the server 2530 can take the form of data packets suitable for transmission between two or more computer processes. The example computing environment 2500 includes a communication framework 2550 that can be used to facilitate communication between the client 2510 and the server 2530. The client 2510 is operably connected to one or more client data repositories 2520 that can be used to store information local to the client 2510. Similarly, the server 2530 is operably connected to one or more server data repositories 2540 that can be used to store information local to the server 2530.

[0232] Fig.26 An example non-limiting apparatus for performing various embodiments described herein is shown in FIG. Fig.26 A non-limiting example of a dual beam system 2610 having a vertically mounted scanning electron microscope (SEM) column and a focused ion beam (FIB) column mounted at an angle of approximately 52 degrees to vertical is illustrated. Such dual beam systems are commercially available, for example, from FEI Company, Hillsboro, Oregon, the assignee of the present application. Although Fig.26 An example of suitable microscope hardware that can implement the various embodiments described herein is shown, but it should be understood that such microscope hardware is non-limiting. In other words, the various embodiments described herein can be implemented in conjunction with any other suitable type of microscopy hardware. Dual beam system 2610 is a non-limiting example of scientific instrument 302 or any other scientific instrument discussed above.

[0233] A scanning electron microscope 2641 and a power supply and control unit 2645 are provided together with the dual beam system 2610. By applying a voltage between the cathode 2652 and the anode 2654, an electron beam 2643 can be emitted from the cathode 2652. The electron beam 2643 can be focused into a fine spot with the aid of a condenser lens 2656 and an objective lens 2658. The electron beam 2643 can be scanned in two dimensions on any suitable sample with the aid of a deflection coil 2660. The operation of the condenser lens 2656, the objective lens 2658 or the deflection coil 2660 can be controlled by the power supply and control unit 2645.

[0234] The electron beam 2643 can be focused onto a substrate 2622, which can be located on a movable XY stage 2625 within a lower chamber 2626. When electrons in the electron beam 2643 strike the substrate 2622, secondary electrons can be emitted. These secondary electrons are detected by a secondary electron detector 2640, as discussed below. A scanning transmission electron microscope (STEM) detector 2662 located below a transmission electron microscope (TEM) sample holder 2624 and the movable XY stage 2625 can collect electrons transmitted through a sample mounted on the TEM sample holder 2624, as discussed above.

[0235] The dual beam system 2610 may also include a focused ion beam (FIB) system 2611, which may include a vacuum chamber having an upper neck portion 2612 in which an ion source 2614 and a focusing column 2616 including an extraction electrode and an electrostatic optical system may be placed. The axis of the focusing column 2616 may be tilted 52 degrees (or any other suitable angular displacement) from the axis of the electron column. The ion column 2612 may include an ion source 2614, an extraction electrode 2615, a focusing element 2617, a deflection element 2620, and a focused ion beam 2618. The focused ion beam 2618 may pass from the ion source 2614 through the focusing column 2616 and between the electrostatic deflection members schematically indicated by the number 2620 toward a substrate 2622, which may include, for example, a semiconductor device on a movable XY stage 2625 located within a lower chamber 2626.

[0236] The movable XY stage 2625 can be moved in the horizontal plane (along the X-axis and Y-axis) and in the vertical plane (along the Z-axis). The movable XY platform 2625 can be tilted about sixty (60) degrees and rotated around the Z-axis. In some embodiments, a separate TEM sample stage (not shown) can be used. Such a TEM sample stage can be moved in the X-axis, Y-axis and Z-axis. The door 2661 can be opened for inserting the substrate 2622 into the movable XY stage 2625, or also for maintaining the internal gas supply reservoir (if used). The door 2661 can be interlocked so that if the system is under vacuum, the door cannot be opened.

[0237] The neck portion 2612 may be evacuated using an ion pump 2668. The chamber 2626 may be evacuated using a turbomolecular and mechanical pumping system 2630 under the control of a vacuum controller 2632. This vacuum system may provide approximately 1×10 -7 Torr and 5×10 -4 If etch assist, etch blocker, or deposition precursor gases are used, the chamber background pressure may be elevated, typically to about 1×10 -5 Entrust.

[0238] The high voltage power supply 2634 can provide appropriate accelerating voltages to the electrodes in the focusing column 2616 for exciting the focused ion beam 2618. When the focused ion beam strikes the substrate 2622, material can be sputtered (i.e., physically ejected) from the sample. Alternatively, the focused ion beam 2618 can decompose a precursor gas to deposit material.

[0239] A high voltage power supply 2634 may be connected to an ion source 2614 (which may be a liquid metal ion source) and appropriate electrodes in an ion beam focusing column 2616 for forming an ion beam 2618 of approximately 1 keV to 60 keV and directing it toward a sample. A deflection controller and amplifier 2636 operating according to a prescribed pattern provided by a pattern generator 2638 may be coupled to a deflection element 2620 (which may be a deflection plate) whereby the focused ion beam 2618 may be manually or automatically controlled to trace a corresponding pattern on an upper surface of a substrate 2622. In some systems, the deflection element 2620 may be placed before the final lens. A beam blanking electrode (not shown) within the ion beam focusing column 2616 may cause the focused ion beam 2618 to impinge on a blanking aperture (not shown) instead of the substrate 2622 when a blanking controller (not shown) applies a blanking voltage to the blanking electrode.

[0240] The ion source 2614 can provide a beam of metal ions, such as gallium. In other examples, the ion source 2614 can be a plasma ion source that extracts ions from a generated plasma. The source can be capable of focusing into a sub-tenth micron wide beam at the substrate 2622 for modifying the substrate 2622 by ion milling, enhanced etching, material deposition, or for imaging the substrate 2622.

[0241] A charged particle detector 2640 (such as an Everhart Thornley or multi-channel board) for detecting secondary ion or electron emission can be connected to a video circuit 2642, which can supply a drive signal to a video monitor 2644 and receive a deflection signal from a system controller 2619. The orientation of the charged particle detector 2640 within the lower chamber 2626 can vary in different embodiments. For example, the charged particle detector 2640 can be coaxial with the ion beam and include an aperture for allowing the ion beam to pass. In other embodiments, the secondary particles can be collected by a final lens and then deflected off-axis for collection.

[0242] The micromanipulator 2647 can precisely move objects within the vacuum chamber. The micromanipulator 2647 can include a precision electric motor 2648 located outside the vacuum chamber to provide X, Y, Z, and theta control of a portion 2649 located within the vacuum chamber. The micromanipulator 2647 can be equipped with different end effectors for manipulating small objects. In various embodiments described herein, the end effector can be a thin probe 2650.

[0243] A gas delivery system 2646 may extend into the lower chamber 2626 for introducing and directing gaseous vapors to the substrate 2622. A suitable gas delivery system 2646 is described in U.S. Pat. No. 5,851,413 to Casella et al., "Gas Delivery Systems for Particle Beam Processing," assigned to the assignee of the present invention. Another gas delivery system is described in U.S. Pat. No. 5,435,850 to Rasmussen, "Gas Injection System," also assigned to the assignee of the present invention. For example, iodine may be delivered to enhance etching, or metal organic compounds may be delivered to deposit metals.

[0244] The system controller 2619 can control the operation of various parts of the dual beam system 2610. Through the system controller 2619, a user can cause the focused ion beam 2618 or the electron beam 2643 to scan in a desired manner through commands entered into any suitable user interface (not shown). Alternatively, the system controller 2619 can control the dual beam system 2610 according to programmed instructions stored in the memory 2621. In various embodiments, any of the one or more software components 315 (e.g., access component 316, model component 318, milling component 320, training component 1302) can be implemented in the system controller 2619 or otherwise executed by the system controller.

[0245] Various embodiments can be systems, methods, devices or computer program products at any possible level of technical detail integration. Computer program products may include computer-readable storage media (or media) with computer-readable program instructions for making the processor execute various aspects of various embodiments. Computer-readable storage media can be tangible devices that can retain and store instructions for use by instruction execution devices. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media may also include the following: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disc read-only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards with instructions recorded on them or raised structures in grooves) and any suitable combination of the above. Computer-readable storage media as used herein should not be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses transmitted through fiber optic cables), or electrical signals transmitted through wires.

[0246] Computer-readable program instructions described herein can be downloaded from computer-readable storage media to corresponding computing / processing equipment or downloaded to external computers or external storage devices via a network (e.g., the Internet, local area network, wide area network or wireless network). The network may include copper transmission cables, optical transmission optical fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers. Network adapter cards or network interfaces in each computing / processing equipment receive computer-readable program instructions from the network, and forward computer-readable program instructions to be stored in the computer-readable storage media in the corresponding computing / processing equipment. Computer-readable program instructions for performing the operation of various embodiments can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcodes, firmware instructions, state setting data, configuration data of integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and process programming languages ​​(e.g., "C" programming languages ​​or similar programming languages). Computer readable program instructions can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider through the Internet). In some embodiments, electronic circuits include, for example, programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), and electronic circuits can be personalized by utilizing the state information of computer readable program instructions, thereby executing computer readable program instructions, to perform various aspects.

[0247] Various aspects are described herein with reference to the flowchart illustration or block diagram of the method, device (system) and computer program product according to various embodiments.It should be understood that each frame in the flowchart illustration or block diagram and the combination of the frames in the flowchart illustration or block diagram can be realized by computer-readable program instructions.These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a component for implementing the function / action specified in one or more frames of the flowchart or block diagram.These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide a computer, a programmable data processing device or other equipment to run in a particular manner, and the computer-readable storage medium storing instructions includes a manufactured product, which includes the instructions of various aspects of the function / action specified in one or more frames of the flowchart or block diagram.Computer-readable program instructions can also be loaded onto a computer, other programmable data processing devices or other devices so that a series of operation actions are performed on a computer, other programmable devices or other devices, thereby generating a computer-implemented process, so that the instructions executed on a computer, other programmable devices or other devices are implemented in one or more frames of the flowchart or block diagram.

[0248] The flow chart and block diagram in the figure illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments.In this regard, each frame in the flow chart or block diagram can represent a module, segment or partial instruction, which contains one or more executable instructions for implementing a specified logical function.In some alternative specific implementations, the function marked in the frame may not occur in the order marked in the figure.For example, two blocks displayed continuously can actually be performed substantially at the same time, or sometimes can be performed in reverse order, depending on the function involved.It should also be noted that each frame in the block diagram or flow chart and the combination of frames in the block diagram or flow chart can be realized by a dedicated hardware system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0249] Although the subject matter has been described above in the general context of computer executable instructions of a computer program product running on a computer, those skilled in the art will recognize that the present disclosure may also or may be implemented in combination with other program modules. Typically, a program module includes routines, programs, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In addition, those skilled in the art will recognize that various aspects can be practiced using other computer system configurations, including single-processor or multi-processor computer systems, small computing devices, large computers, and computers, handheld computing devices (e.g., PDAs, phones), microprocessor-based or programmable consumer or industrial electronics, etc. The illustrated aspects may also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected via a communication network. However, some aspects of the present disclosure (if not all aspects) may be practiced on a stand-alone computer. In a distributed computing environment, a program module may be located in both a local memory storage device and a remote memory storage device.

[0250] The terms "component", "system", "platform", "interface", etc. used in this application may refer to or may include computer-related entities or entities related to an operating machine having one or more specific functions. The entities disclosed herein may be hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, a processor, an object, an executable program, a thread of execution, a program, or a computer running on a processor. As an example, both an application running on a server and a server may be a component. One or more components may reside within a process or an execution thread, and a component may be located on a computer or distributed between two or more computers. In another example, the corresponding components may be executed from various computer-readable media having various data structures stored thereon. These components may communicate via a local or remote process, such as according to a signal having one or more data packets (for example, data from a component interacts with another component in a local system, a distributed system, or interacts with other systems via a network such as the Internet). As another example, a component may be a device having a specific function provided by a mechanical part operated by an electrical or electronic circuit, which is operated by a software or firmware application executed by a processor. In this case, the processor may be inside or outside the device and may execute at least a portion of the software or firmware application. As yet another example, a component can be a device that provides a specific functionality through an electronic component without mechanical parts, where the electronic component may include a processor or other means for executing software or firmware that at least partially imparts the functionality to the electronic component. In one aspect, the component can emulate the electronic component via a virtual machine within a cloud computing system, for example.

[0251] In addition, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" is intended to mean any natural inclusive arrangement. That is, if X employs A; X employs B; or X employs both A and B, then "X uses A or B" is satisfied under any of the foregoing examples. As used herein, the term "and / or" is intended to have the same meaning as "or". In addition, unless otherwise specified or clear from the context that it involves a singular form, the article "one" as used in should generally be interpreted to mean "one or more". As used herein, the term "example" or "exemplary" is used to mean as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited to such examples. In addition, any aspect or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to exclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.

[0252] The content disclosed herein describes non-limiting examples. For ease of description or explanation, the various parts disclosed herein use the terms "each," "each," or "all" when discussing various examples. The usage of terms such as "each," "each," or "all" is not restrictive. In other words, when the content disclosed herein provides a description of "each," "all," or "all" applicable to a particular object or component, it should be understood that this is only a non-limiting example, and it should also be understood that in various other examples, such descriptions can be applicable to descriptions of "each," "all," or "all" less than the particular object or component.

[0253] As used in this specification, the term "processor" may refer to substantially any computing processing unit or device, including but not limited to a single-core processor; a single processor with software multithreaded execution capability; a multi-core processor; a multi-core processor with software multithreaded execution capability; a multi-core processor with hardware multithreading technology; a parallel platform; and a parallel platform with distributed shared memory. In addition, a processor may refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof designed to perform the functions described herein. In addition, the processor may utilize nanoscale architectures, such as (but not limited to) transistors, switches, and gates based on molecules and quantum dots to optimize space usage or enhance the performance of user equipment. The processor may also be implemented as a combination of computing processing units. In the present disclosure, terms such as "repository", "storage device", "data repository", "data storage device", "database", and substantially any other information storage component related to the operation and function of the component are used to refer to a "memory component", an entity embodied in a "memory", or a component including a memory. It should be understood that the memory and / or memory component can be a volatile memory or a non-volatile memory, or can include both volatile memory and non-volatile memory. By way of example and not limitation, the non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable ROM (EEPROM), a flash memory, or a non-volatile random access memory (RAM) (e.g., a ferroelectric RAM (FeRAM)). The volatile memory may include a RAM, which may, for example, act as an external cache memory. By way of example and not limitation, RAM takes a variety of forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). In addition, the memory components of the systems disclosed herein or the computer-implemented methods are intended to include, but are not limited to, these and any other suitable types of memory.

[0254] The foregoing includes only examples of systems and computer-implemented methods. Of course, it is not possible to describe every conceivable combination of components or computer-implemented methods for purposes of describing the present disclosure, but many further combinations and permutations of the present disclosure are possible. In addition, with respect to the use of the terms "including," "having," "having," and the like in the detailed description, claims, appendices, and drawings, these terms are intended to be inclusive in a manner similar to that in which the term "comprising" is interpreted when used as a transitional word in a claim.

[0255] Descriptions of various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed herein. Many modifications and variations are apparent without departing from the scope and spirit of the embodiments. The terminology used herein is selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies on the market, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0256] Various non-limiting aspects are described in the following examples.

[0257] Embodiment 1: A system may include: a processor that executes computer executable components stored in a non-transitory computer readable memory, wherein the computer executable components include: an access component that is capable of accessing an image captured by a scientific instrument, wherein the image is capable of depicting a cross-section of a thin slice; and a model component that is capable of generating a classification label for the cross-section via execution of a graph weighted neural network, wherein the classification label is capable of indicating to which of multiple defined classes the cross-section belongs.

[0258] Embodiment 2: A system according to any one of the preceding embodiments that can be implemented, wherein the computer executable component further comprises: a milling component that can command the scientific instrument to incrementally mill the cut surface of the slice in response to the classification label indicating that the cut surface does not belong to a target class among the multiple defined classes.

[0259] Embodiment 3: A system according to any one of the preceding embodiments that can be implemented, wherein the graph weighted neural network can include a deep learning neural network and a class neighborhood graph, wherein the nodes of the class neighborhood graph can represent corresponding classes among the multiple defined classes, and wherein the weighted edges of the class neighborhood graph can indicate which classes among the multiple defined classes are physically adjacent and non-adjacent to each other within the slice along the milling direction.

[0260] Embodiment 4: A system according to any one of the preceding embodiments that can be implemented, wherein the deep learning neural network is capable of receiving the image or a portion of the image as input and generating an original classification label as output, wherein the original classification label is capable of indicating an original probability that the slice belongs to a corresponding class among the multiple defined classes, wherein a graph component is capable of multiplying the original probability by the corresponding weighted edges indicated in the class neighborhood graph and normalizing the resulting product to obtain a modified probability, and wherein the modified probabilities are capable of jointly forming the classification label.

[0261] Embodiment 5: A system according to any one of the preceding embodiments that can be implemented, wherein the class neighborhood graph can include a first node, one or more second nodes, and one or more third nodes, wherein the first node can represent a first class among the multiple defined classes, wherein the one or more second nodes can respectively represent one or more second classes among the multiple defined classes that are expected to be physically adjacent to the first class along the milling direction, wherein the one or more third nodes can respectively represent one or more third classes among the multiple defined classes that are expected to be physically non-adjacent to the first class along the milling direction, wherein the first node can have a high-weight edge with itself, wherein the first node can have one or more medium-weight edges with the one or more second nodes, and wherein the first node can have one or more low-weight edges with the one or more third nodes.

[0262] Embodiment 6: According to the system described in any preceding embodiment that can be implemented, the class neighborhood graph can be generated based on the collection and annotation of training images for the deep learning neural network.

[0263] Embodiment 7: The system according to any preceding embodiment capable of being implemented, wherein the class neighborhood graph can be generated based on a computer-aided design file associated with the sheet.

[0264] Embodiment 8: A system according to any of the preceding embodiments that can be implemented, wherein the thin film can include a field effect transistor, and wherein the plurality of defined classes can include one or more first classes associated with a gate of the field effect transistor, one or more second classes associated with a source or drain of the field effect transistor, or one or more third classes associated with a fin of the field effect transistor.

[0265] In various embodiments, any one or more combinations of Examples 1 to 8 may be implemented.

[0266] Embodiment 9: A computer-implemented method may include: accessing, by a device operatively coupled to a processor, an image captured by a scientific instrument, wherein the image may depict a cross-section of a slice; and generating, by the device and via execution of a graph weighted neural network, a classification label for the cross-section, wherein the classification label may indicate to which of a plurality of defined classes the cross-section belongs.

[0267] Embodiment 10: The computer-implemented method according to any of the preceding embodiments that can be implemented further includes: commanding the scientific instrument to incrementally mill the cut surface of the slice by the device and in response to the classification label indicating that the cut surface does not belong to a target class among the multiple defined classes.

[0268] Embodiment 11: A computer-implemented method according to any of the preceding embodiments that can be implemented, wherein the graph weighted neural network can include a deep learning neural network and a class neighborhood graph, wherein the nodes of the class neighborhood graph can represent corresponding classes among the multiple defined classes, and wherein the weighted edges of the class neighborhood graph can indicate which classes of the multiple defined classes are physically adjacent and non-adjacent to each other within the slice along the milling direction.

[0269] Embodiment 12: A computer-implemented method according to any one of the preceding embodiments that can be implemented, wherein the deep learning neural network is capable of receiving the image or a portion of the image as input and generating an original classification label as output, wherein the original classification label is capable of indicating an original probability that the slice belongs to a corresponding class among the multiple defined classes, wherein the device is capable of multiplying the original probability by the corresponding weighted edges indicated in the class neighborhood graph and normalizing the resulting product to obtain a modified probability, and wherein the modified probabilities are capable of collectively forming the classification label.

[0270] Embodiment 13: A computer-implemented method according to any one of the preceding embodiments that can be implemented, wherein the class neighborhood graph can include a first node, one or more second nodes, and one or more third nodes, wherein the first node can represent a first class among the multiple defined classes, wherein the one or more second nodes can respectively represent one or more second classes among the multiple defined classes that are expected to be physically adjacent to the first class along the milling direction, wherein the one or more third nodes can respectively represent one or more third classes among the multiple defined classes that are expected to be physically non-adjacent to the first class along the milling direction, wherein the first node can have a high-weight edge with itself, wherein the first node can have one or more medium-weight edges with the one or more second nodes, and wherein the first node can have one or more low-weight edges with the one or more third nodes.

[0271] Embodiment 14: A computer-implemented method according to any preceding embodiment that can be implemented, wherein the class neighborhood graph can be generated based on the collection and annotation of training images for the deep learning neural network.

[0272] Embodiment 15: The computer-implemented method of any preceding embodiment as may be implemented, wherein the class neighborhood graph may be generated based on a computer-aided design file associated with the sheet.

[0273] Embodiment 16: A computer-implemented method according to any of the preceding embodiments that can be implemented, wherein the thin film can include a field effect transistor, and wherein the plurality of defined classes can include one or more first classes associated with a gate of the field effect transistor, one or more second classes associated with a source or drain of the field effect transistor, or one or more third classes associated with a fin of the field effect transistor.

[0274] In various embodiments, any one or more combinations of Examples 9 to 16 may be implemented.

[0275] Embodiment 17: A computer program product for facilitating thin slice endpoint positioning via a graph weighted neural network can include a non-transitory computer readable memory, wherein the non-transitory computer readable memory contains program instructions. In various aspects, the program instructions can be executed by a processor to cause the processor to: access an image captured by a dual-beam charged particle microscope, wherein the image can depict a cross-section of a thin slice; generate a classification label for the cross-section via execution of the graph weighted neural network, wherein the classification label can indicate which of a plurality of structures the cross-section contains; and in response to the classification label indicating that the cross-section does not contain a target structure, command the dual-beam charged particle microscope to incrementally mill the cross-section of the thin slice.

[0276] Embodiment 18: A computer program product according to any one of the preceding embodiments that can be implemented, wherein the graph weighted neural network can include a deep learning neural network and a neighborhood graph, wherein the nodes of the neighborhood graph can represent corresponding structures of the plurality of structures, and wherein the weighted edges of the neighborhood graph can indicate which structures of the plurality of structures are physically adjacent and non-adjacent to each other within the slice along the milling direction.

[0277] Embodiment 19: A computer program product according to any one of the preceding embodiments that can be implemented, wherein the deep learning neural network is capable of receiving the image or a portion of the image as input and generating an original classification label as output, wherein the original classification label is capable of indicating an original probability that the section contains a corresponding structure among the multiple structures, wherein the processor is capable of multiplying the original probability by the corresponding weighted edges indicated in the neighborhood graph and normalizing the resulting product to obtain a modified probability, and wherein the modified probabilities are capable of jointly forming the classification label.

[0278] Embodiment 20: A computer program product according to any one of the preceding embodiments that can be implemented, wherein the neighborhood graph can include a first node, one or more second nodes and one or more third nodes, wherein the first node can represent a first structure among the multiple structures, wherein the one or more second nodes can respectively represent one or more second structures among the multiple structures that are expected to be physically adjacent to the first structure along the milling direction, wherein the one or more third nodes can respectively represent one or more third structures among the multiple structures that are expected to be physically not adjacent to the first structure along the milling direction, wherein the first node can have a high-weight edge with itself, wherein the first node can have one or more medium-weight edges with the one or more second nodes, and wherein the first node can have one or more low-weight edges with the one or more third nodes.

[0279] In various embodiments, any one or more combinations of Examples 17 to 20 may be implemented.

[0280] In various embodiments, any one or more combinations of Examples 1 to 20 may be implemented.

Claims

1. A system comprising: a processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components include: an access component that accesses an image captured by a scientific instrument, wherein the image depicts a cross-section of the slice; and A model component generates a classification label for the slice via execution of a graph weighted neural network, wherein the classification label indicates to which class of a plurality of defined classes the slice belongs.

2. The system of claim 1, wherein the computer executable components further comprise: A milling assembly commands the scientific instrument to incrementally mill the slice facet of the slice in response to the classification label indicating that the slice facet does not belong to a target class among the plurality of defined classes.

3. The system of claim 1 , wherein the graph weighted neural network comprises a deep learning neural network and a class neighborhood graph, wherein nodes of the class neighborhood graph represent corresponding classes of the plurality of defined classes, and wherein weighted edges of the class neighborhood graph indicate which classes of the plurality of defined classes are physically adjacent and non-adjacent to each other within the slice along a milling direction.

4. The system of claim 3, wherein the deep learning neural network receives the image or a portion of the image as input and produces a raw classification label as output, wherein the raw classification label indicates a raw probability that the slice belongs to a corresponding class among the plurality of defined classes, wherein a graph component multiplies the raw probabilities by corresponding weighted edges indicated in the class neighborhood graph and normalizes the resulting products to yield modified probabilities, and wherein the modified probabilities collectively form the classification label.

5. A system according to claim 3, wherein the class neighborhood graph includes a first node, one or more second nodes and one or more third nodes, wherein the first node represents a first class among the multiple defined classes, wherein the one or more second nodes respectively represent one or more second classes among the multiple defined classes that are known to be physically adjacent to the first class along the milling direction, wherein the one or more third nodes respectively represent one or more third classes among the multiple defined classes that are known to be physically not adjacent to the first class along the milling direction, wherein the first node has a high-weight edge with itself, wherein the first node has one or more medium-weight edges with the one or more second nodes, and wherein the first node has one or more low-weight edges with the one or more third nodes.

6. The system of claim 3, wherein the class neighborhood graph is generated based on collection and annotation of training images for the deep learning neural network.

7. The system of claim 3, wherein the class neighborhood graph is generated based on a computer-aided design file associated with the sheet.

8. The system of claim 1 , wherein the wafer comprises a field effect transistor, and wherein the plurality of defined classes comprises one or more first classes associated with a gate of the field effect transistor, one or more second classes associated with a source or drain of the field effect transistor, or one or more third classes associated with a fin of the field effect transistor.

9. A computer-implemented method comprising: accessing, by a device operatively coupled to the processor, an image captured by the scientific instrument, wherein the image depicts a cross-section of the thin slice; as well as A classification label for the slice is generated by the device and via execution of a graph weighted neural network, wherein the classification label indicates to which of a plurality of defined classes the slice belongs.

10. The computer-implemented method of claim 9, further comprising: The scientific instrument is commanded, by the device and in response to the classification label indicating that the cut surface does not belong to a target class among the plurality of defined classes, to incrementally mill the cut surface of the slice.

11. The computer-implemented method of claim 9, wherein the graph weighted neural network comprises a deep learning neural network and a class neighborhood graph, wherein nodes of the class neighborhood graph represent corresponding classes of the plurality of defined classes, and wherein weighted edges of the class neighborhood graph indicate which classes of the plurality of defined classes are and are not physically adjacent to each other within the slice along a milling direction.

12. A computer-implemented method according to claim 11, wherein the deep learning neural network receives the image or a portion of the image as input and produces a raw classification label as output, wherein the raw classification label indicates a raw probability that the slice belongs to a corresponding class among the multiple defined classes, wherein the device multiplies the raw probability by the corresponding weighted edge indicated in the class neighborhood graph and normalizes the resulting product to obtain a modified probability, and wherein the modified probabilities collectively form the classification label.

13. A computer-implemented method according to claim 11, wherein the class neighborhood graph includes a first node, one or more second nodes and one or more third nodes, wherein the first node represents a first class among the multiple defined classes, wherein the one or more second nodes respectively represent one or more second classes among the multiple defined classes that are known to be physically adjacent to the first class along the milling direction, wherein the one or more third nodes respectively represent one or more third classes among the multiple defined classes that are known to be physically not adjacent to the first class along the milling direction, wherein the first node has a high-weight edge with itself, wherein the first node has one or more medium-weight edges with the one or more second nodes, and wherein the first node has one or more low-weight edges with the one or more third nodes.

14. The computer-implemented method of claim 11, wherein the class neighborhood graph is generated based on a collection and annotation of training images for the deep learning neural network.

15. The computer-implemented method of claim 11, wherein the class neighborhood graph is generated based on a computer-aided design file associated with the sheet.

16. The computer-implemented method of claim 9, wherein the slice comprises a field effect transistor, and wherein the plurality of defined classes comprises one or more first classes associated with a gate of the field effect transistor, one or more second classes associated with a source or drain of the field effect transistor, or one or more third classes associated with a fin of the field effect transistor.

17. A computer program product for facilitating sheet endpoint localization via a graph weighted neural network, the computer program product comprising a non-transitory computer readable memory having program instructions embodied therein, the program instructions executable by a processor to cause the processor to: accessing images captured by a dual-beam charged particle microscope, wherein the images depict a cross-section of the thin slice; generating a classification label for the slice by executing a graph weighted neural network, wherein the classification label indicates which structure of a plurality of structures the slice includes; and The dual-beam charged-particle microscope is commanded to incrementally mill the section of the lamella in response to the classification tag indicating that the section does not contain a target structure.

18. The computer program product of claim 17, wherein the graph weighted neural network comprises a deep learning neural network and a neighborhood graph, wherein nodes of the neighborhood graph represent corresponding structures of the plurality of structures, and wherein weighted edges of the neighborhood graph indicate which structures of the plurality of structures are and are not physically adjacent to each other within the slice along a milling direction.

19. A computer program product according to claim 18, wherein the deep learning neural network receives the image or a portion of the image as input and produces a raw classification label as output, wherein the raw classification label indicates a raw probability that the slice contains a corresponding structure in the plurality of structures, wherein the processor multiplies the raw probability by the corresponding weighted edge indicated in the neighborhood graph and normalizes the resulting product to obtain a modified probability, and wherein the modified probabilities collectively form the classification label.

20. The computer program product of claim 18, wherein the neighborhood graph comprises a first node, one or more second nodes, and one or more third nodes, wherein the first node represents a first structure among the plurality of structures, wherein the one or more second nodes respectively represent one or more second structures among the plurality of structures that are known to be physically adjacent to the first structure along the milling direction, wherein the one or more third nodes respectively represent one or more third structures among the plurality of structures that are known to be not physically adjacent to the first structure along the milling direction, wherein the first node has a high-weight edge with itself, wherein the first node has one or more medium-weight edges with the one or more second nodes, and wherein the first node has one or more low-weight edges with the one or more third nodes.

Images