Device line understanding using synthetic templates

Abstract

The invention relates to a device line understanding using a synthetic template. Systems, components, methods, and algorithms for generating template information are described. A computer-implemented method includes: receiving microscope image data; detecting features in the microscope image data; receiving feature template data describing a packet of features including the feature; detecting the packet in the microscope image data based at least in part on the feature template data; generating template information describing the group; and outputting the template information.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application S / N 18 / 341,685, filed on June 26, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Embodiments of the present disclosure relate to charged particle microscope systems and algorithms and methods for their operation. In particular, some embodiments relate to techniques for system automation. Background Art

[0004] Semiconductor manufacturing utilizes charged particle microscopy (such as transmission electron microscopy) as part of quality control. In an illustrative example, a sample of semiconductor devices (e.g., integrated circuits) is extracted from a wafer or wafer portion (e.g., a diced wafer) and examined in a transmission electron microscope. Microscope images of so-called "device lines" (referring to a linear arrangement of individual devices such as transistors, capacitors, etc.) can be used to assess manufacturing errors and / or design errors. Thus, one aspect of quality control of semiconductor devices includes determining whether a manufactured device conforms to a device design, wherein the individual devices are present at a given location on a wafer and / or wafer portion in an expected number and sequence.

[0005] Typically, quality control of semiconductors is at least partially, if not entirely, manual. Sample preparation, including positioning and extracting thin slices, generating microscope image data, and analyzing image data to determine whether the manufactured device conforms to the device design, as well as other operations, is performed by skilled technicians. Efforts to automate various aspects of the quality control process, including image processing automation, face significant challenges. For example, defect detection relies on correctly identifying regions of interest in samples, generating images of regions of interest, and processing images via feature detection and recognition, pattern matching, and the like. Each of these operations is challenging for machine vision systems, which typically perform poorly in distinguishing IC devices from backgrounds, are prone to errors when samples are inverted in microscopes, and / or are relatively inflexible in analyzing device line samples that deviate from a predefined sequence. Therefore, improved data processing techniques, methods, and / or algorithms are needed for quality control of semiconductor manufacturing. Summary of the invention

[0006] In one aspect, a computer-implemented method includes: receiving microscope image data; detecting a feature in the microscope image data; receiving feature template data describing a grouping of features including the feature; detecting the grouping in the microscope image data based at least in part on the feature template data; generating template information describing the grouping; and outputting the template information.

[0007] Detecting a feature in the microscope image data may include: inputting at least a portion of the microscope image data into a model configured to detect the feature and generate coordinate information describing the location of the feature in the microscope image data as an output of the model. The location of the feature may correspond to a set of coordinates for the center of mass of the feature. The model may include a convolutional neural network trained to input a portion of the microscope image data and output coordinate information. The feature may be a first feature, the model may be a first model, the portion may be a first portion, the coordinate information may be first coordinate information, and the position may be a first position. The microscope image data may include a second feature. The method may further include: inputting at least a second portion of the microscope image data into a second model configured to detect a second feature and generate second coordinate information describing a second location of the second feature in the microscope image data as an output of the second model.

[0008] The feature template data may describe a sequence of a plurality of features including a first feature. Detecting the grouping may include convolving the feature template data with the coordinate information and detecting an instance of the grouping in the microscope image data based at least in part on a position of the feature relative to at least a subset of the plurality of features in the sequence. Detecting the instance of the grouping may include determining a first ordering of the first feature in the sequence and determining that the first ordering of the first feature and a second ordering of the second feature in the sequence match the feature template data.

[0009] The feature template data may include feature multiplicity information and feature order information. Detecting groupings in the image data may include detecting instances of groupings having reversed feature order. The features may form at least a portion of a device in an integrated circuit. The microscope image data may include an image generated by a charged particle microscope. The microscope image data may further include coordinate metadata mapping pixels of the image data to locations on a sample.

[0010] Outputting the template information may include sending the template information to a charged particle microscope configured to generate image data based at least in part on the template information. The template information may include a location of the grouping in the microscope image data. The location of the grouping may correspond to a vertex of a bounding box surrounding the grouping.

[0011] In one aspect, one or more non-transitory machine-readable media store instructions that, when executed by a machine, cause the machine to perform the operations of the method of the aforementioned aspects.

[0012] In one aspect, a system includes: an analytical instrument configured to generate image data; and a computing device operably connected to the analytical instrument and configured to receive the image data from the analytical instrument. The computing device may be configured to include the medium and / or perform the operations of the method of the foregoing aspects. The analytical instrument may be or include a charged particle microscope. The computing device may be an instrument PC, a client computing device, or one or more servers. The computing device may be configured to receive the image data from the analytical instrument via one or more networks.

[0013] Embodiments of the present disclosure also include systems, components and methods according to the aforementioned aspects. The terms and expressions that have been adopted are used as descriptive terms rather than restrictive terms, and when using such terms and expressions, it is not intended to exclude any equivalents of the features shown and described or parts thereof, but it should be recognized that various modifications are possible within the scope of the claimed subject matter. Therefore, it should be understood that although the subject matter claimed by the present disclosure has been specifically disclosed through embodiments and optional features, those skilled in the art may modify and change the concepts disclosed herein, and such modifications and changes are considered to be within the scope of the present disclosure as defined by the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The foregoing aspects of the present disclosure and many of the attendant advantages will become more readily appreciated when the following detailed description is taken in conjunction with the accompanying drawings.

[0015] Figure 1 is a schematic diagram illustrating an example charged particle microscope system according to some embodiments of the present disclosure.

[0016] Figure 2 is a schematic diagram illustrating an example analysis system for generating microscopy image data and template information according to some embodiments of the present disclosure.

[0017] Figure 3 is a flow chart of an example process for generating template information using image data according to some embodiments of the present disclosure.

[0018] Figure 4 is a block diagram of an example object hierarchy describing template information according to some embodiments of the present disclosure.

[0019] Figure 5 is a block flow diagram illustrating example data flow involved in generating template information according to some embodiments of the present disclosure.

[0020] Figure 6 is a schematic diagram illustrating an example convolution technique for generating template information according to some embodiments of the present disclosure.

[0021] In the drawings, unless otherwise specified, the same reference numerals represent the same components in various views. Where appropriate, not all instances of components need to be labeled to reduce clutter in the drawings. The drawings are not necessarily drawn to scale, with emphasis instead on illustrating the principles being described. DETAILED DESCRIPTION

[0022] Although illustrative embodiments have been described and described, it should be understood that various changes may be made therein without departing from the spirit and scope of the present disclosure. In the following paragraphs, embodiments of charged particle microscope systems, components, and methods for device detection and identification in integrated circuit samples are described. In order to simplify the description, the embodiments of the present disclosure focus on CMOS device samples imaged in a transmission electron microscope (TEM). To this end, the embodiments are not limited to such samples or instruments, but rather envision analytical instrument systems in which the analysis of microstructure and / or nanostructure features can benefit from robust automation (e.g., operation without human intervention) and / or pseudo-automation (e.g., operation with limited human intervention). In illustrative examples, the technology of the present disclosure can be applied to image data derived from microbial processes (e.g., gene sequencing output, fluorescence microscope images, etc.), metastructure materials (e.g., quality control of hidden geometric structures for manufacturing) and / or applied to large-scale imaging of artificial or natural structures (e.g., survey data, such as hyperspectral imaging). Similarly, while embodiments of the present disclosure focus on TEM, additional and / or alternative systems are contemplated, including but not limited to scanning electron microscopes (SEMs), scanning transmission electron microscopes (STEMs), STEMs in SEMs, atomic force microscopes (AFMs), scanning capacitance microscopes (SCMs), ion microscopes (IMs), optical microscopes, confocal microscopes, fluorescence microscopes, hyperspectral imaging, and the like, wherein the instrument is used to generate image data representing structures or other features that at least partially conform to a pattern.

[0023] Embodiments of the present disclosure include systems, methods, algorithms, and non-transitory media storing computer-readable instructions for generating template information from image data. In an illustrative example, a method may include: receiving microscope image data; detecting features in the microscope image data; receiving feature template data describing a grouping of features including the feature; detecting a grouping in the microscope image data based at least in part on the feature template data; generating template information for the grouping; and outputting the template information. As described with reference to the following embodiments, the hierarchical structure of the template data allows the system of the present disclosure to efficiently process microscope image data, detect features in the arrangement corresponding to a design or other characteristic pattern, determine an area of ​​interest in a semiconductor sample for imaging, and guide the system to generate image data including the area of ​​interest. In this way, an analytical instrument system can perform template-driven image generation with improved performance in terms of time and computing resource requirements with reduced human interaction, while also reducing the level of technical complexity required of human operators of the instrument system.

[0024] Figure 1 1 is a schematic diagram illustrating an example charged particle microscope system 100 according to some embodiments of the present disclosure. The example system 100 may include: one or more instrument systems 105, one or more instrument PCs (IPCs) 110, one or more client computing devices (client PCs) 115, and / or one or more servers 120. The various components of the example system 100 may communicate via one or more networks 125 and / or via a direct connection (e.g., a USB type connection, Bluetooth, WiFi, Ethernet, etc.). In some embodiments, one or more components of the example system 100 are omitted. For example, embodiments of the present disclosure may omit the client PC 115 and / or the server 120.

[0025] The instrument system 105 may include components for analyzing a material sample according to one or more measurement modalities, which is facilitated by the configuration of the instrument 105 and software, tools, etc. available on the IPC 110, the client PC 115, and / or the server 120. For example, the client PC 115 and / or the server 120 may host a software application configured to implement one or more processing operations using data generated by the instrument 105. In this manner, the software application may be hosted locally on a single device and / or on a distributed computing system, and may direct the operation of the instrument 105 based at least in part on template information generated from image data generated by the instrument 105.

[0026] The client computing device 115 may be or include a general purpose computing device (e.g., a laptop, tablet computer, smart phone, desktop computer, etc.) and / or a dedicated computing device. The server 120 may be or include one or more local and / or remote network-connected machines, including processing, storage and / or communication components. In an illustrative example, the server 120 may be co-located with the instrument 105 in a physical location (e.g., a building, campus, or other location) and may communicate with one or more components of the instrument 105. The client PC 115 may be located at a first physical location different from a second physical location of the instrument 105. To this end, the constituent elements of the example system 100 may be co-located to store large data sets generated by the instrument system 105 and to reduce data transmission delays during relatively high network latency periods, or, for example, when the first physical location is physically far away from the second physical location (e.g., on different continents or on different coasts of the same continent).

[0027] IPC 110 may include a general-purpose or special-purpose computing device. For example, embodiments include a PC configured for user interaction (e.g., having a display, user-interaction peripherals, and a user interface), a PC dedicated to coordinating the operation of instrument 105 without direct user interaction without user interface components (also referred to as a "dedicated" PC), and / or a computing board incorporated into instrument 105 or otherwise operably coupled to the instrument. The computing board may include components similar to a dedicated PC, wherein power circuitry and / or input-output components may be shared with instrument 105.

[0028] Figure 2 is a schematic diagram illustrating an example analysis system 200 for generating microscope image data 220 and template information 240 according to some embodiments of the present disclosure. In the illustrated embodiment, the example analysis system 200 includes a TEM sample 205 as an example of a sample form used in the instrument 105 of the example system 100. In this manner, the microscope image data 220 includes a microscope image formed by passing a charged particle beam 210 through a sample 215 that is at least partially transparent to electrons.

[0029] Figure 2The example analysis system 200 of TEM is shown with a TEM sample such that the charged particle beam 210 is an electron beam. In some embodiments, the charged particle beam 210 is an ion beam or includes an ion beam (e.g., extracted from an ion source such as a liquid metal ion source, a plasma-based ion source, etc.). In addition, the charged particle beam 210 is shown as passing through the sample 215, so that the image data 220 corresponds to an image constructed with detector data generated from the primary charged particles of the charged particle beam 210 (e.g., primary electrons in the case of a TEM system). In some embodiments, the charged particle beam 210 is focused onto the sample 215, so that the image data 220 may include backscattered electrons, backscattered ions, secondary electrons, and / or secondary ion detector data. In this case, the sample 215 may be almost completely or completely impermeable to the charged particles, such as a bulk material sample or a sample having a thickness such that electron absorption and / or backscattering exceeds electron transmission.

[0030] Image data 220 shows an area of ​​sample 215 that includes features 225 and 230 arranged in an arrangement. The arrangement can be a linear arrangement and / or a nonlinear arrangement. In the example of a semiconductor device, features 225 and 230 can be arranged in a "device line" that includes multiple instances of features 225 and 230 formed during a CMOS manufacturing process. Features 225 and 230 can be or include at least a portion of various devices that constitute a part of an integrated circuit, such as a transistor, a capacitor, a via, etc. In some cases, the same device can be detected as more than one feature type based at least in part on changes in the characteristics of the device. For example, size can be used to classify a device into one of a set of feature types (e.g., small 227, medium 228, and large 229). Similarly, other characteristics such as material composition, sub-features, etc. (e.g., in one or more pre-processing operations) can be used to define multiple feature types.

[0031] The image data 220 may include metadata, including, but not limited to, sample information, system parameters, and / or spatial / coordinate information. The spatial / coordinate metadata may map pixels in the image data 220 to a set of coordinates of locations on the sample 205. In some cases, the coordinates may reference a stage control scheme, such as a multi-axis sample holder (e.g., three spatial directions and tilt). In this manner, features 225 and 230 may be referenced by locations on the sample 205 and in the image data 220. For example, a feature may be referenced by the location of the center of mass of the feature in the image data 220 and / or on the sample 205. The extent, center of mass, and other geometric properties of various features may be determined based at least in part on a segmentation process applied to the image data 220. Additionally or alternatively, the size, extent, and / or center of mass of various features may be determined based at least in part on detection of one or more edges, vertices, or other aspects of the feature, and a predefined center of mass may be referenced from one or more edges, vertices, or other aspects of the feature. For example, the vertex between the top edge and the side edge of a given feature may be referenced to define the center of mass based on the design specifications of the given feature. As referenced Figure 3 As described in more detail, the location of the centroid can be referenced to a bounding box used to define a region of interest (ROI) for further microscopic examination and / or microanalysis (eg, by imaging or probing).

[0032] To this end, the techniques of the present disclosure may include a process applied to image data 220, according to which the observed arrangement of features 225 and / or 230 may be compared to feature template data (e.g., Figures 4 to 5 The characteristic template data 405 is compared as a method to guide the microscopic inspection and microanalysis of the sample 205. Figure 2 The features 225 and 230 shown in the figure are schematic in nature and do not represent the shape or size of an actual semiconductor device. Instead, the image data 220 is provided as an illustrative example of features arranged in the sample 215, and the template information 240 can be generated from the image data. In addition, although the image data 220 includes two types of features, the techniques of the present invention can be applied to image data describing samples including more or fewer types of features.

[0033] Template information 240 may include data of features 225 and 230 (including feature sequences), such as the number of instances of first feature type 225 and the number of instances of second feature type 230 in grouping 235. Thus, template information 240 may include a sequence of features (e.g., in order of corresponding positions in an arrangement), where grouping 235 corresponds to the sequence of features. Figure 2In the example of , the template corresponds to a sequence of features in the device line, for which the grouping 235 includes a first instance 225-1 of the first feature, an instance of the second feature 230, and a second instance 225-2 of the first feature. The image data 220 may describe multiple instances of the grouping 235, and not all features 225 and / or 230 may belong to a grouping instance (e.g., the third instance 225-3 of the first feature). As shown in reference Figure 3 Described in more detail, the template information 240 may be generated by: processing the image data 220 using an algorithm to detect features in the image data 220; identifying the arrangement of features in groups associated with feature template data; and generating information describing the location and sequence of the groups to be used to study the structure of the sample.

[0034] Figure 3 is a flowchart of an example process 300 for generating template information using image data according to some embodiments of the present disclosure. One or more operations constituting the example process 300 may be performed by a computer system and / or an analytical instrument (e.g., Figure 1 The components of process 300 may be operably connected to other machines and / or additional systems or subsystems (including but not limited to characterization systems, network infrastructure, databases, controllers, relays, power systems and / or user interface devices) to perform and / or initiate. Therefore, the operations may be stored in one or more machine-readable media as machine-executable instructions, which, when executed by a computer system, may cause the computer system to perform at least a portion of the constituent operations of process 300. The constituent operations of process 300 may be performed before, between and / or after operations omitted in this specification (such as sample and / or instrument preparation, operations occurring before operation 305, etc.), which form a process for processing samples to generate such as Figure 2 To this end, in some embodiments, the operations of the example process 300 may be omitted, repeated, reordered, and / or replaced.

[0035] At operation 305, the example process includes receiving image data (e.g., Figure 2 Receiving image data may include various sub-operations associated with data storage and retrieval. For example, the image data may be stored in a local storage system and / or a distributed storage system so that a computing device (e.g., Figure 1 The IPC 110, client PC 115, server 120, etc.) can request, retrieve, or otherwise access image data for a given sample. In some embodiments, receiving image data includes generating image data, such as reference Figure 2To this end, the example process 300 may be implemented by one or more computing systems concurrently with imaging a sample in a charged particle microscope, for example, as part of a semiconductor quality control scheme. In the illustrative example, the microscope image data (e.g., Figure 2 Image data 220) may be generated and retrieved from a charged particle microscope (e.g., Figure 1 instrument 105) to a computing device or other machine that is implementing the operations of the example process 300.

[0036] At operation 310, the example process includes detecting one or more features in the image data. Feature detection may include enabling detection and / or recognition of features in the image data based at least in part on the ability to detect and / or recognize features (e.g., Figure 2 Detecting one or more features may include inputting at least a portion of the microscope image data into a model configured to detect the feature and generate coordinate information describing the location of the feature in the microscope image data as an output of the model. Figures 4 to 5 As described in more detail, the model may be or include a method configured to input at least a portion of the microscopy image data and output information describing the order, location and / or size of given features and information that can be used to detect groupings in the microscopy image data (e.g., Figure 2 The model structure may include a rule-based model, a feature detection algorithm (e.g., edge detection based on a Sobel filter, texture analysis based on a Gabor filter, etc.), a neural network-based model (e.g., a convolutional neural network), a pixel-based classification method, an image block-based image classification method, etc.

[0037] In some embodiments, multiple models are used to detect multiple features in the image data. The models may be configured to detect corresponding feature types so that for a given number of features to be detected in the microscope image data, the same number of models are prepared. For example, two convolutional neural network models may be trained as part of the preparatory operations of the example process 300, such that the first model detects a first feature type and the second model detects a second feature type. To this end, detecting one or more features in the image data may include inputting at least a second portion of the microscope image data to a second model configured to detect a second feature and generating second coordinate information describing a second position of the second feature in the microscope image data as an output of the second model. In some cases, a complete microscope image data set is processed that is input to multiple models. However, in some cases, operation 310 may include sub-operations, such as segmentation and pre-processing (e.g., feature agnostic) to divide the features into smaller data sets, thereby reducing the amount of data processed by the model. The smaller data sets may be marked with grid references or other indexes as a method of tracking the ordering of features in the sequence, such as references Figure 2 as well as Figures 5 and 6 Described in more detail.

[0038] At operation 315, the example process includes receiving feature template data. Figure 2 as well as Figures 4 to 6 Described in more detail, feature template data (e.g., Figure 4 The feature template data 405 may describe multiple features (e.g., Figure 2 For example, the feature template data may describe the spatial arrangement of multiple features using relative coordinates that can be compared to detected features in the microscope image data. In a simpler example, the feature template data may describe a sequence of multiple features, where the features are assigned sequence positions (e.g., using integer ordering). In this way, the feature template data may describe a sample (e.g., Figure 2 The feature template data may include metadata such as feature type, feature size, relative position of the center of mass, or other information that can be used to map the arrangement of features in the microscope image data to the feature template. In some embodiments, the feature template data includes feature multiplicity information and feature order information. The feature multiplicity may describe the number of instances of a given feature, while the feature order information may describe the relative ordering of the features in the sequence. In some embodiments, the template may include multiple groupings (e.g., Figure 2 235), as referenced Figure 4 Described in more detail.

[0039] At operation 320, the example process includes detecting one or more packets. A packet (e.g., Figure 2 The grouping 235) may include multiple instances of a feature in the microscope image data. For example, in one arrangement, the grouping may include one or more instances of a first feature, one or more instances of a second feature, and one or more instances of a third feature. Detecting the grouping may include one or more methods using the data generated at operation 310. For example, the ranking data generated by feature detection may be convolved with the feature template data, as described in reference to Figure 6 Detecting one or more groupings may include detecting instances of groupings in the microscopy image data based at least in part on positions of features relative to at least a subset of a plurality of features in a sequence. In this manner, detecting instances of groupings in the microscopy image data may include determining a first ordering of a first feature in the sequence, and determining that the first ordering of the first feature and the second ordering of the second feature in the sequence match feature template data. For example, detecting a grouping may include finding a matching sequence of features in an arrangement (e.g., a first feature, followed by a second feature, followed by a first feature (e.g., Figure 2 and Figure 6 ) can be identified as a group).

[0040] The group detection may be based at least in part on the feature multiplicity and / or feature order, as described with reference to operation 315. For some analytical instrument systems (e.g., TEM systems, STEM systems, etc.), the sample (e.g., Figure 2 The sample 215) can be introduced into the sample holder in more than one orientation. For example, in a TEM holder, the lamella can define two broad faces, either of which can be oriented toward the incident electron beam (e.g., Figure 2 210) orientation of the charged particle beam. In this way, the feature template data can describe order information that can be directly mapped to observed features detected in the microscope image data or mapped to an inverted arrangement (e.g., mirror inversion) caused by the inversion of the sample in the analytical instrument system. Therefore, operation 320 may include an instance of detecting grouping based on the inverted feature order. In one example, in a linear arrangement, the inverted feature order of a template having two first features followed by a second feature will be a sequence of one second feature followed by two first features.

[0041] At operation 325, the example process includes generating template information. The template information may include metadata derived from the microscope image data based at least in part on the grouping detected at operation 320. For example, the template information derived from a given grouping may include the coordinates of the grouping, which may include, without limitation, the location of the grouping in the microscope image data and / or in the sample. The location of the grouping may be described using a centroid or other substantially centered location of the grouping, an extent of the grouping (e.g., a set of four corner coordinates, a bounding box, etc.), a vertex or other peripheral coordinates and / or an origin of a bounding box surrounding the grouping, contour data, etc. Advantageously, using a centroid or other substantially centered location to define the location of the grouping may allow instructions to be generated for an analytical instrument so that the instrument generates new image data for which a region of interest is centered, substantially centered, or includes the grouping. In this manner, the grouping may be detected at a relatively low magnification in the microscope image data and analyzed in an automatic or pseudo-automatic method at a relatively high magnification in the new microscope image data.

[0042] At operation 330, the example process includes outputting the template information. Operation 330 may include sending the template information to a charged particle microscope configured to generate image data based at least in part on the template information, as described with reference to operation 325. In some embodiments, operation 330 includes storing the template information, which may include transferring the template information between computing devices (e.g., via Figure 1 Operation 330 may also include generating visualization data based at least in part on the template information. The visualization data may include instructions configured to modify a display or other device to present the template information to a computing device (e.g., Figure 1 Instructions as part of a user environment (e.g., a browser or application environment on a display) on the IPC 110 and / or client PC 115 of the computer. In an illustrative example, visualization data may be generated to modify the display to overlay a bounding box on an area of ​​the microscope image data, wherein the bounding box represents an area of ​​interest substantially centered on one or more groups. In some embodiments, operation 330 includes outputting feature and / or grouping information such that the visualization data may include feature rankings, feature identifiers, feature centroids, grouping centroids, number of grouping instances, and / or template order information, as well as other types of information. Thus, operation 330 may include outputting information from multiple hierarchical levels of features, groups, and templates, which allows samples to be systematically and efficiently interrogated through automated and / or pseudo-automated techniques.

[0043] Figure 44 is a block diagram of an example object hierarchy 400 describing template information according to some embodiments of the present disclosure. The example hierarchy 400 includes a top-level object associated with feature template data 405, the top-level object including data for a plurality of clusters 410, which in turn includes one or more features 415. The example hierarchy 400 includes in the image data (e.g., Figure 2 The template information may include three object levels as described in the image data 220 of FIG. 1 , but the template information may include more or fewer object levels. For example, a group (e.g., Figure 2 The grouping 235) may represent a template or cluster 410.

[0044] Features 415 may have one or more types, such as reference Figure 2 As described in more detail herein, each feature 415 type may be associated with a model configured to detect the feature 415 type in image data. The model may generate metadata for detecting clusters 410 and / or templates 405 in image data, as described in reference to FIG. Figure 3 as well as Figures 5 and 6 410. Similarly, clusters 410 may represent combinations of features and / or composite features described in device designs, intended arrangements, etc., such that detecting clusters 410 may be computationally simpler than detecting constituent features 415. In an illustrative example, in addition to or in lieu of detecting the individual features that make up a cluster, a device line may include clusters of continuous features such that the outer boundaries of a cluster are suitable for detecting the cluster. Thus, one or more models may be configured to detect clusters 410 in image data.

[0045] Figure 5 is a flowchart of an example data processing workflow 500 involved in generating template information according to some embodiments of the present disclosure. The data processing workflow 500 includes a flow chart applied to a data processing flow generated by an analytical instrument 105 (e.g., Figure 1 The operations of the example workflow 500 are shown as a series of operations that are not associated with a specific instrument, computing device, or machine, as a way to illustrate the generation of multiple data types and / or data structures at various points in the example workflow 500. In addition, the example workflow 500 includes at least one processing operation of data 505 generated by an instrument 501 of the example system 100. The operations of the example workflow 500 are shown as a series of operations that are not associated with a specific instrument, computing device, or machine, as a way to illustrate the generation of multiple data types and / or data structures at various points in the example workflow 500. In addition, the example workflow 500 includes at least one processing operation of data 505 generated by a plurality of computing devices (e.g., Figure 1 A subset of the processing operations may be performed on a server 120) and / or in parallel by processing data 505 on a multi-core processor.

[0046] Data 505 may be or include image data (e.g., Figure 2 of the microscope image data 220), but may also include a description of the sample (e.g., Figure 2For example, the data may be optical image data generated using an optical microscope and / or camera, spectral mapping image data generated using spatially resolved spectroscopy, force microscopy data that maps electronic properties of a sample surface in two or more dimensions, etc.

[0047] Data 505 may be distributed in whole or in part to one or more models 510, which are configured to detect features in data 505 (e.g., Figure 2 Features 225 and 230 and Figure 4 415) and / or groupings of features (e.g., Figure 4 Cluster 410). As shown in reference Figure 3 As described in more detail, data 505 may be segmented by one or more pre-processing operations as a method of improving parallelization, and model 510 may be or include various types of image processing models and / or algorithms (e.g., CNN-type machine learning models, image patch-based processing algorithms, etc.). In this manner, the output from model 510 may be or include data 515 describing feature information (such as order, location in image data, and / or location in samples). Data 515 output from model 510 may be combined to generate aggregated data 520 and provided to template detection subprocess 530. As described in reference Figure 6 As described in more detail, one or more template detection subprocesses 530 may be configured to map feature template data 405 to output data 515 and / or aggregate data 520. In some cases, the template detection subprocess 530 may be implemented as an algorithm configured to generate template information 535, such as reference 534. Figure 3 Described in more detail.

[0048] Template information 535 may include template location, ROI data, feature sequence data (such as feature multiplicity and order information observed in a sample), template instance information, or information that allows an analysis system (e.g., Figure 1 The template information 535 may then be provided as input to an output subprocess 540, which may be or include one or more algorithms for generating visualization data, sample positioning instructions, beam direction instructions, detector operation instructions, instrument operation parameters, etc. In the illustrative example, the output subprocess 540 may generate instruction data 545 to direct the instrument 501 to generate additional data based at least in part on the template information 535 generated using the data 505.

[0049] Figure 66 is a schematic diagram illustrating an example convolution technique 600 for generating template information according to some embodiments of the present disclosure. The example technique 600 includes mapping feature template data 405 to output data 515 and / or aggregate data 520 describing feature ordering and identity information of device line samples. In the example technique 600, the feature template data 405 describes the following groups (e.g., Figure 2 The convolution method includes mapping the feature template data 405 onto a sequence of features in the aggregated data 520 to determine whether the feature matches the template, followed by incrementing the template by one level if the feature does not match the template, or by incrementing the template by a template length if the feature does match the template. For one or more of the templates being detected (e.g., each template or fewer of the templates), an instance number may be assigned, as well as various location / position data relative to the reference template in the image data and / or on the sample. Figure 6 In the example, the data includes the centroid position of the XY coordinates in the reference image or on the sample.

[0050] In some embodiments, additional and / or alternative techniques are used to map feature template data 405 to aggregate data 520 as part of generating feature information 535 for a sample. For example, a technique for measuring the error or distance (e.g., mean square error) between a template and the measured feature can be used to generate feature information 535. Similarly, template matching methods can include outlier removal (e.g., random sample consistency techniques) to improve matching in noisy data. Advantageously, template matching methods can be extended to higher dimensions. Embodiments of the present disclosure include two-dimensional convolutions (e.g., along a second axis in an array of features).

[0051] In the foregoing description, various embodiments have been described. For the purpose of explanation, specific configurations and details have been set forth in order to provide a thorough understanding of the embodiments. However, it will also be apparent to those skilled in the art that these embodiments can be practiced without these specific details. In addition, in order not to obscure the described embodiments, well-known features may have been omitted or simplified. Although the example embodiments described herein are centered on a microscope system, and in particular on a charged particle microscope system, these are meant to be non-restrictive illustrative embodiments. The embodiments of the present disclosure are not limited to such embodiments, but are intended to propose an analytical instrument system in which a wide array of material samples can be analyzed to determine chemical, biological, physical, structural or other properties, as well as other aspects, including but not limited to materials, chemistry, physics, electronic structure, etc., trace element composition, etc.

[0052] Some embodiments of the present disclosure include a system having one or more data processors and / or logic circuits. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions, which, when executed on one or more data processors and / or logic circuits, cause the one or more data processors and / or logic circuits to perform part or all of one or more methods disclosed herein and / or part or all of one or more processes and workflows. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product including instructions configured to cause one or more data processors and / or logic circuits to perform part or all of one or more methods disclosed herein and / or part or all of one or more processes.

[0053] The terms and expressions employed herein are used as terms of description rather than limitation, and when such terms and expressions are used, it is not intended to exclude any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the claims. Therefore, it should be understood that while the present disclosure includes specific embodiments and optional features, modifications and changes to the concepts disclosed herein may be made by those skilled in the art, and such modifications and changes are deemed to be within the scope of the appended claims.

[0054] When a term is used without explicit definition, it is understood that unless the term has a special and / or particular meaning in the field of charged particle microscopy systems or other related fields, the term refers to the ordinary meaning of the word. The term "about" or "substantially" is used to indicate a deviation from the stated property, wherein the deviation has little or no effect on the corresponding function, property or attribute of the described structure. In the illustrated example where one dimensional parameter is described as "substantially equal to" another dimensional parameter, the term "substantially" is intended to reflect that the two parameters being compared may be unequal within permissible limits (such as manufacturing tolerances or confidence intervals inherent to system operation). Similarly, where a geometric parameter (such as alignment or angular orientation) is described as "approximately" perpendicular, "substantially" perpendicular, or "substantially" parallel, the term "about" or "substantially" is intended to reflect that the alignment or angular orientation may differ from the precisely stated condition (e.g., not precisely perpendicular) within permissible limits. For dimensional values ​​(such as diameter, length, width, etc.), the term "about" may be understood to describe a deviation of up to ±10% from the stated value. For example, a size of "about 10 mm" may describe a size from 9 mm to 11 mm.

[0055] This description provides exemplary embodiments, and is not intended to limit the scope, applicability or configuration of the present disclosure. On the contrary, the subsequent description of the exemplary embodiments will provide enabling descriptions for realizing various embodiments to those skilled in the art. It should be understood that, without departing from the spirit and scope set forth in the appended claims, various changes may be made to the functions and arrangements of the elements. Specific details are given in the description to provide a thorough understanding of the embodiments. However, it should be understood that the embodiments may be put into practice without these specific details. For example, the specific system components, systems, processes and other elements of the present disclosure may be shown in schematic form, or omitted from the diagram, so as to avoid obscuring the embodiments due to unnecessary details. In other cases, well-known circuits, processes, components, structures and / or technologies may be shown without unnecessary details.

Claims

1. A computer-implemented method, the method comprising: receiving microscope image data; detecting features in the microscope image data; receiving feature template data describing a group of features including the feature; detecting the grouping in the microscope image data based at least in part on the feature template data; generating template information describing the grouping; as well as The template information is output.

2. The computer-implemented method of claim 1 , wherein detecting the feature in the microscope image data comprises: inputting at least a portion of the microscope image data into a model configured to detect the feature; as well as Coordinate information describing the location of the feature in the microscope image data is generated as an output of the model. 3 . The computer-implemented method of claim 2 , wherein the position of the feature corresponds to a set of coordinates of a center of mass of the feature.

4. The computer-implemented method of claim 2, wherein the model comprises a convolutional neural network trained to input the portion of the microscope image data and output the coordinate information.

5. The computer-implemented method of claim 2, wherein the feature is a first feature, the model is a first model, the part is a first part, the coordinate information is first coordinate information, and the position is a first position, and wherein the microscope image data includes a second feature, the method further comprising: inputting at least a second portion of the microscope image data into a second model configured to detect the second feature; as well as Second coordinate information describing a second position of the second feature in the microscope image data is generated as an output of the second model.

6. The computer-implemented method of claim 2, wherein the feature is a first feature, wherein the feature template data describes a sequence of a plurality of features including the first feature, and wherein detecting the grouping comprises: Convolving the feature template data with the coordinate information; as well as Instances of the grouping are detected in the microscopy image data based at least in part on positions of the features relative to at least a subset of the plurality of features in the sequence.

7. The computer-implemented method of claim 6, wherein said detecting said instance of said packet comprises: determining a first ranking of the first features in the sequence; as well as It is determined that the first ordering of the first features and the second ordering of second features in the sequence match the feature template data.

8. The computer-implemented method of claim 1, wherein the feature template data includes feature multiplicity information and feature order information.

9. The computer-implemented method of claim 8, wherein detecting the grouping in the image data comprises detecting instances of the grouping having a reversed feature order.

10. The computer-implemented method of claim 1, wherein the feature forms at least a portion of a device in an integrated circuit.

11. The computer-implemented method of claim 1 , wherein the microscope image data comprises an image generated by a charged particle microscope.

12. The computer-implemented method of claim 11, wherein the microscope image data further comprises coordinate metadata mapping pixels of the image data to locations on a sample.

13. The computer-implemented method of claim 1, wherein outputting the template information comprises sending the template information to a charged particle microscope configured to generate image data based at least in part on the template information.

14. The computer-implemented method of claim 1, wherein the template information includes locations of the groups in the microscopy image data.

15. The computer-implemented method of claim 14, wherein the positions of the groupings correspond to vertices of a bounding box surrounding the groupings.

16. One or more non-transitory machine-readable media storing instructions that, when executed by a machine, cause the machine to perform operations comprising: receiving microscope image data; detecting features in the microscope image data; receiving feature template data describing a group of features including the feature; detecting the grouping in the microscope image data based at least in part on the feature template data; generating template information describing the grouping; as well as The template information is output.

17. The medium of claim 16, wherein detecting the feature in the microscope image data comprises: inputting at least a portion of the microscope image data into a model configured to detect the feature; as well as Coordinate information describing the location of the feature in the microscope image data is generated as an output of the model.

18. The medium of claim 17, wherein the feature is a first feature, the model is a first model, the part is a first part, the coordinate information is first coordinate information, and the position is a first position, and wherein the microscope image data includes a second feature, the operations further comprising: inputting at least a second portion of the microscope image data into a second model configured to detect the second feature; as well as Second coordinate information describing a second position of the second feature in the microscope image data is generated as an output of the second model.

19. The medium of claim 16, wherein outputting the template information comprises sending the template information to a charged particle microscope configured to generate image data based at least in part on the template information.

20. The medium of claim 17, wherein the feature is a first feature, wherein the feature template data describes a sequence of a plurality of features including the first feature, and wherein detecting the packet comprises: Convolving the feature template data with the coordinate information; as well as Instances of the grouping are detected in the microscopy image data based at least in part on positions of the features relative to at least a subset of the plurality of features in the sequence.

Images