Training an artificial neural network using simulated sample images

Abstract

The present invention describes a technique for training an artificial neural network (ANN) using simulated sample images. The simulated sample images are generated based on a data model. The data model describes the characteristics of a crystalline material and the characteristics of one or more defect types. The data model does not include any image data. The simulated sample images are input as training data into a training algorithm to generate an artificial neural network (ANN) for identifying defects in crystalline materials. After training, the ANN analyzes captured sample images to identify the defects shown therein.

Description

Technical Field

[0001] The present disclosure relates to an artificial neural network for sample image analysis. Specifically, the present disclosure relates to training an artificial neural network using simulated sample images. Background Art

[0002] Microscopy is the field of technology that uses microscopes to better observe objects that are difficult to see with the naked eye. Different branches of microscopy include, for example: optical microscopy, charged particle (electron and / or ion) microscopy, and scanning probe microscopy.

[0003] Charged particle microscopy involves the use of an accelerated beam of charged particles as a source of illumination. Types of electron microscopes include, for example: transmission electron microscope, scanning electron microscope, scanning transmission electron microscope, and focused ion beam microscope.

[0004] The components of a transmission electron microscope (TEM) include: an electron optical column, a vacuum system, necessary electronics (lens sources for focusing and deflecting the beam and a high voltage generator for the electron source), and control software. The electron optical column includes an electron gun on one end and an observation device (such as a camera) on the other end. The electron beam is emitted from the electron gun and passes through a thin sample to transmit electrons that are collected, focused, and projected onto the observation device. The entire electron path from the gun to the camera is under vacuum.

[0005] Similar to the TEM, the components of a scanning electron microscope (SEM) include: an electron optical column, a vacuum system, necessary electronics (lens sources for focusing and deflecting the beam and a high voltage generator for the electron source), and control software. The electron gun is located at one end of the electron optical column. The sample is located at the other end of the electron column. The electron beam from the electron gun is focused into a fine spot on the sample surface. The electron beam is scanned in a rectangular raster over the sample. The intensity of various signals generated by the interaction between the beam electrons and the sample is measured and stored in a computer memory. The stored values ​​are then mapped as changes in brightness on an image display.

[0006] Scanning transmission electron microscopy (STEM) is similar to TEM in that an image is formed by electrons passing through a sufficiently thin sample. However, instead of capturing the image all at once, STEM focuses the electron beam onto a tiny spot and then scans the electron beam over the sample in a raster illumination system.

[0007] A focused ion beam microscope (FIB microscope) is similar to a SEM, however, a FIB microscope uses an ion beam rather than an electron beam. Examples of ion beam sources include a liquid metal ion source (LMIS), such as a gallium ion source.

[0008] The microscope is associated with various configurable microscope parameters. Examples of microscope parameters for a SEM include: accelerating voltage (the voltage at which electrons are accelerated as they pass through the electron optical column); the convergence angle of the electron beam; beam current; spot size (the diameter of the beam spot on the sample); dwell time; resolution; backscattered electron (BSE) detector collection angle; and collection efficiency. Different values ​​of the various microscope parameters result in images of different quality and properties. For example, a higher magnification requires a smaller spot size. A higher signal-to-noise ratio and contrast resolution require a larger beam current. However, reducing the spot size also reduces the beam current.

[0009] A sample is an object (or part of an object) that is studied and / or analyzed using a microscope. In the semiconductor field, examples of samples include crystalline materials and semiconductor materials. Crystalline materials include a target layer and a substrate layer. The target layer can be deposited or grown on the substrate layer via epitaxy. Examples of crystalline materials include silicon germanium (SiGe) as a target layer and silicon (Si) as a substrate layer. A common problem with such crystalline materials is the generation of structural defects such as dislocations and stacking faults. Microscopes can be used to observe any defects within the crystalline material. One method for capturing images of crystalline materials is electron channel contrast imaging (ECCI).

[0010] Artificial neural networks (ANNs) can be used to identify defects within crystalline materials as shown in sample images. Artificial neural networks are a class of machine learning or artificial intelligence systems. The artificial neural network is trained on a set of training data, the training data including sample images, the sample images showing crystalline materials including defects. Each sample image is labeled with information associated with the defect, such as the number of defects in the sample image, the defect type of the defect in the sample image, and / or the location of the defect in the sample image. A larger set of training data generally improves the accuracy of the artificial neural network. However, generating training data may involve a time-consuming process. First, it is necessary to produce crystalline materials including defects. However, some types of defects are rare and / or difficult to generate. Secondly, a microscope is used to capture sample images of the crystalline material. Third, the captured sample images are labeled with information associated with the defects.

[0011] The approaches described in this section are approaches that could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] These embodiments are shown in the accompanying drawings by way of example and not by way of limitation. It should be noted that reference to "one" or "an" embodiment in the present disclosure does not necessarily refer to the same embodiment, and it refers to at least one embodiment. In the accompanying drawings:

[0013] Figure 1A An example artificial neural network training system is shown in accordance with one or more embodiments;

[0014] Figure 1B illustrates an example defective crystal data model according to one or more embodiments;

[0015] Figure 1C illustrates an example process flow for training an artificial neural network using simulated sample images according to one or more embodiments;

[0016] Figure 2A B to B illustrate a set of example operations for training an artificial neural network using simulated sample images according to one or more embodiments;

[0017] Figure 3 illustrates an example of simulated sample images for training an artificial neural network according to one or more embodiments; and

[0018] Figure 4 A block diagram illustrating a computer system in accordance with one or more embodiments is shown. DETAILED DESCRIPTION

[0019] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in different embodiments. In some examples, well-known structures and devices are described with reference to block diagram form to avoid unnecessarily obscuring the present invention.

[0020] 1. Overview

[0021] 2. Artificial Neural Network Training System Architecture

[0022] 3. Training artificial neural network using simulated sample images

[0023] 4. Example Embodiments

[0024] 5. Hardware Overview

[0025] 6. Miscellaneous; Extension

[0026] 1. Overview

[0027] One or more embodiments include training an artificial neural network (ANN) to identify defects in a crystalline material shown in a sample image. The captured sample image is generated based on a view of the sample using a microscope. In contrast, a simulated sample image is generated based on a data model. The data model describes the characteristics of the crystalline material and the characteristics of one or more defect types. The data model does not include any image data. The simulated sample image is input as training data into a training algorithm to generate an artificial neural network (ANN) for identifying defects in crystalline materials. After training, the artificial neural network analyzes the captured sample images to identify the defects shown therein.

[0028] One or more embodiments include selectively generating simulated sample images to generate a modified artificial neural network. An artificial neural network trained based on the simulated sample images analyzes captured sample images to identify defects displayed therein. Determine defect recognition rates for various defect types achieved by the artificial neural network. Generate additional simulated sample images including defect types associated with lower defect recognition rates. Do not generate additional simulated sample images including defect types associated with higher defect recognition rates. The additional simulated sample images are input into a training algorithm as training data to generate a modified artificial neural network. In other embodiments, simulated sample images associated with lower defect recognition rates and simulated sample images associated with higher defect recognition rates are generated and input into a training algorithm as training data to generate a modified artificial neural network.

[0029] By using simulated sample images, a large amount of training data for artificial neural networks can be efficiently generated. Using captured sample images as training data requires first generating crystalline materials including defects. However, some types of defects are rare and / or difficult to generate. Instead, simulated sample images showing defects of multiple defect types can be generated based on data models describing different defect types. Further, the types of defects shown in the simulated sample images can be specifically customized so that the training of the artificial neural network can be targeted to specific defect types.

[0030] One or more embodiments described in the specification and / or recited in the claims may not be included in this summary.

[0031] 2. Artificial Neural Network Training System Architecture

[0032] Figure 1A An example artificial neural network (ANN) training system is shown in accordance with one or more embodiments. Figure 1A As shown in FIG. 1 , the system 100 includes a data model generator 102, a simulation model 104, a training algorithm 106, an artificial neural network 108, and a data repository 110. In one or more embodiments, the system 100 may include a data model generator 102, a simulation model 104, a training algorithm 106, an artificial neural network 108, and a data repository 110. Figure 1AMore or fewer components may be used than shown. Figure 1A The components shown in may be local to each other or remote from each other. Figure 1A The components shown in the can be implemented in software and / or hardware. Each component can be distributed on multiple applications and / or machines. Multiple components can be combined in one application and / or machine. The operation described with respect to one component can be performed by another component instead.

[0033] In one or more embodiments, the data repository 110 is any type of storage unit and / or device for storing data (e.g., a file system, a database, a collection of tables, or any other storage mechanism). Further, the data repository 110 may include multiple different storage units and / or devices. The multiple different storage units and / or devices may be of the same type or not of the same type, or located at the same physical location. Further, the data repository 110 may be implemented or executed on the same computing system as the data model generator 102, the simulation model 104, the training algorithm 106, and / or the artificial neural network 108. Alternatively or additionally, the data repository 110 may be implemented or executed on a computing system separate from the data model generator 102, the simulation model 104, the training algorithm 106, and / or the artificial neural network 108. The data repository 110 may be coupled to the data model generator 102, the simulation model 104, the training algorithm 106, and / or the artificial neural network 108 via a direct connection or via a network communication.

[0034] Information describing one or more defective crystal data models 112, one or more simulated sample images 114, and one or more captured sample images 115 may be implemented across any component within system 100. However, for purposes of clarity and explanation, this information is shown within data repository 110.

[0035] In one or more embodiments, the defect crystal data model 112 is a description of a crystal material including one or more defects. The defect crystal data model 112 may include data represented in text and / or digital format. The defect crystal data model 112 may include information organized as name-value pairs. The defect crystal data model 112 may be stored as one or more data files. The data files may be stored in, for example, JSON (JavaScript Object Notation) format and / or HDF5 (Hierarchical Data Format) format.

[0036] The defective crystal data model 112 does not include any image data or data represented in an image format. Image data is data that can be rasterized into a pixel grid. Image data formats include raster, vector, uncompressed, compressed formats. Examples of raster formats include JPEG (Joint Photographic Experts Group), EXIF ​​(Exchangeable Image File Format), TIFF (Tagged Image File Format), GIF (Graphics Interchange Format), BMP, and PNG (Portable Network Image Format). Examples of vector formats include CGM (Computer Graphics Metafile), Gerber, and SVG (Scalable Vector Graphics).

[0037] Figure 1B An example defect crystal data model is shown. Figures 1A to 1C Components labeled with the same number in the figure refer to similar components. As shown in the figure, the defect crystal data model 112 includes one or more target layer features 116, one or more substrate layer features 118, one or more defect type features 120, one or more defect location features 122, one or more microscope features 124 and one or more noise features 126.

[0038] In one or more embodiments, the target layer features 116 are features of a target layer of a crystalline material. The target layer exhibits a periodic crystal structure. The positions of atoms or molecules occur at repeated fixed distances, determined by unit cell parameters. An example of a target layer is silicon germanium (SiGe).

[0039] The target layer features 116 may be expressed using text and / or digital data. The target layer features 116 do not include any image data. The target layer features 116 may include, for example, the following pieces of information associated with the target layer:

[0040] (a) Crystal system (such as cubic, tetragonal, hexagonal, orthorhombic, rhombohedral, monoclinic or triclinic);

[0041] (b) lattice parameters (such as the Bravais lattice);

[0042] (c) space group number;

[0043] (d) atomic coordinates;

[0044] (e) placeholder parameters; and

[0045] (f) Debye-Waller factor

[0046] In one or more embodiments, substrate layer features 118 are features of a substrate layer of a crystalline material. The substrate layer exhibits a periodic crystal structure. The positions of atoms or molecules occur at repeated fixed distances, determined by unit cell parameters. An example of a substrate layer is silicon (Si).

[0047] The substrate layer features 118 may be expressed using text and / or digital data. The substrate layer features 118 do not include any image data. The substrate layer features 118 may include, for example, the following pieces of information associated with the substrate layer:

[0048] (a) Crystal system (such as cubic, tetragonal, hexagonal, orthorhombic, rhombohedral, monoclinic or triclinic);

[0049] (b) lattice parameters (such as the Bravais lattice);

[0050] (c) space group number;

[0051] (d) atomic coordinates;

[0052] (e) placeholder parameters; and

[0053] (f) Debye-Waller factor

[0054] In one or more embodiments, the defect type signature 120 is a signature of a structural defect within a crystalline material. The defect disrupts the repeating configuration of atoms or molecules in the target layer and / or substrate layer. The defect may appear as, for example, a point defect, a line defect, and / or a surface defect. As an example, a Burgers vector may represent the magnitude and direction of a lattice distortion caused by a dislocation in a crystalline material.

[0055] In one or more embodiments, the defect location feature 122 is a description of the location of a defect within the crystal material. The defect location can be described relative to an edge (or other location) of a sample image of the crystal material. The defect location can be expressed as an xy coordinate relative to an edge (or other location) of a sample image of the crystal material. One or more defect locations can be specified, indicating the corresponding location of one or more defects within the crystal material.

[0056] In one or more embodiments, microscope features 124 are features of a microscope that is simulated to generate a sample image of a crystalline material.

[0057] The microscope feature 124 may indicate the type of microscope being used. Examples of microscope types include electron microscopes, ion microscopes, scanning electron microscopes, transmission electron microscopes, and scanning transmission electron microscopes.

[0058] Additionally or alternatively, the microscope features 124 may indicate microscope parameters configured for the microscope. Examples of microscope parameters for a scanning electron microscope include: accelerating voltage (the voltage at which electrons are accelerated as they pass through the electron optical column); the convergence angle of the electron beam; the beam current; the spot size (the diameter of the beam spot on the sample); the dwell time; the resolution; the backscattered electron (BSE) detector collection angle; and the collection efficiency. Other types of microscopes may use additional and / or alternative microscope parameters. Different values ​​of various microscope parameters result in images of different qualities and properties. For example, a higher magnification requires a smaller spot size. A higher signal-to-noise ratio and contrast resolution require a larger beam current. However, reducing the spot size also reduces the beam current.

[0059] In one or more embodiments, noise feature 126 is a noise feature simulated in the sample image. The noise may be due to, for example, interference with the microscope used to capture the sample image, dust or other impurities on the microscope and / or the sample, and / or other unknown noise sources.

[0060] Return to reference Figure 1A In one or more embodiments, the captured sample image 115 is a sample image generated based on a view of the sample using a microscope. In contrast, the simulated sample image 114 is an image of the sample simulated based on the defect crystal data model 112. The simulated sample image 114 of the sample is generated without using a microscope to observe the sample. The specimen may or may not exist. The sample may even include materials that cannot be produced using current manufacturing methods.

[0061] In one or more embodiments, the data model generator 102 refers to hardware and / or software configured to generate a defect crystal data model 112. The data model generator 102 can generate multiple defect crystal data models that are associated with the same crystal material and defect type, but are associated with different microscopic features and / or noise features. Additionally or alternatively, the data model generator 102 can modify a specific defect crystal data model associated with a specific crystal material and a specific defect type. The data model generator 102 can modify the microscopic features and / or noise features of a specific defect crystal data model to generate multiple defect crystal data models. Figure 2A 1 to 3 describe an example of an operation for generating the defective crystal data model 112 .

[0062] In one or more embodiments, the simulation model 104 refers to hardware and / or software configured to generate a simulated sample image 114 using the defect crystal data model 112. The simulation model 104 includes an image capture simulator and an optional merging module.

[0063] The image capture simulator accepts a defect crystal data model as input. The image capture simulator generates an original simulated sample image as output. An example of an image capture simulator is EMsoft. EMSOFT: GENERAL USER AND DEVELOPMENT MANUAL (Program Manual, Version 3.1), March 25, 2017, hereby submitted to Appendix A and incorporated by reference.

[0064] The merging module merges the original simulated sample image and the captured sample image to generate a final simulated sample image. As an example, a portion of the original simulated sample image that includes simulated defects can be identified. A portion of the captured sample image that includes actual captured noise can be identified. The identified portion of the original simulated sample image and the identified portion of the captured sample image can be merged to generate a final simulated sample image.

[0065] In one or more embodiments, the training algorithm 106 refers to hardware and / or software configured to train the artificial neural network 108. Training the artificial neural network 108 includes adjusting weights associated with connections between artificial neurons within the artificial neural network 108. The training algorithm 106 may include back propagation, support vector machines (SVM), pattern recognition, Bayesian statistics, and / or other supervised or unsupervised learning methods.

[0066] The training algorithm 106 accepts training data as input. The training data includes sample data to be analyzed by the artificial neural network 108. The sample data is labeled with the desired output of the artificial neural network 108. As an example, the training data can be sample images showing crystalline materials including defects. Each sample image can be labeled with a corresponding defect type for the defect shown in the sample image. The training data can be input into the training algorithm to generate an artificial neural network that recognizes defects in crystalline materials as shown in the sample images.

[0067] The training algorithm 106 can be applied to successive sets of training data to generate a modified and / or improved artificial neural network 108. The training algorithm 106 can be initially applied to a small set of training data to generate an initial artificial neural network 108. Subsequently, additional training data can be collected and / or obtained. The training algorithm 106 can be applied to a larger set of training data (including additional training data). Thus, the training algorithm 106 generates a modified and / or improved artificial neural network 108.

[0068] An example of the operation of training an artificial neural network using simulated sample images 114 will be referred to below. Figure 2A To B for description.

[0069] In one or more embodiments, the artificial neural network 108 (also referred to as a deep learning system) is a machine learning computing system. The artificial neural network 108 includes a collection of connected units or nodes called artificial neurons. Each connection transmits a signal from one artificial neuron to another. The artificial neuron processes the received signal to transmit another signal to another artificial neuron. Artificial neurons can be clustered into layers. Different layers can perform different kinds of transformations on their inputs.

[0070] One type of artificial neural network is a convolutional neural network. Convolutional neural networks are designed to process data that comes in the form of multiple arrays, such as a color image consisting of three two-dimensional arrays containing pixel intensities in three color channels. An example architecture of a traditional neural network is structured as a series of stages. The first few stages consist of two types of layers: convolutional layers and pooling layers. Convolutional layers apply a convolution operation to the input, passing the result to the next layer. Convolution simulates the response of a single neuron to a visual stimulus. Pooling layers combine the outputs of a cluster of neurons at one layer into a single neuron in the next layer. For example, max pooling uses the maximum value of each of the clusters of neurons at the previous layer.

[0071] In one or more embodiments, the artificial neural network 108 is configured to identify defects within the crystalline material as shown in the sample image. Identifying defects within the crystalline material includes classifying the defects into a specific defect type.

[0072] In one or more embodiments, the data model generator 102, the simulation model 104, the training algorithm 106, and / or the artificial neural network 108 are implemented on one or more digital devices. The term "digital device" generally refers to any hardware device including a processor. A digital device may refer to a physical device that executes an application or a virtual machine. Examples of digital devices include computers, tablet computers, laptops, desktops, netbooks, servers, network servers, network policy servers, proxy servers, general-purpose machines, special-purpose hardware devices, mainframes, televisions, content receivers, printers, mobile phones, smart phones, and / or personal digital assistants (PDAs).

[0073] Figure 1C An example process flow for training an artificial neural network using simulated sample images according to one or more embodiments is shown. Figures 1A to 1C Components labeled with the same numbers in the drawings refer to similar components.

[0074] As shown, the data model generator 102 generates one or more defect crystal data models 112. A particular defect crystal data model 112 may be generated based on a user specification of a crystal material and / or defect type. Additional defect crystal data models 112 may be generated by modifying a particular defect crystal data model 112. Additional defect crystal data models 112 may be associated with the same crystal material and / or defect type as a particular defect crystal data model 112, but may be associated with different microscopic features and / or noise features.

[0075] The defect crystal data model 112 is input into the simulation model 104. The image capture simulator within the simulation model 104 generates an original simulated sample image based on the defect crystal data model 112. Optionally, a merging module within the simulation model 104 merges a portion of the original simulated sample image and a portion of the captured sample image 115a. For example, the merging module merges (a) a portion of the original simulated sample image showing simulated defects, and (b) a portion of the scanned sample image showing spots of dust on the sample. As used herein, a "simulated sample image" may refer to (a) an original simulated sample image generated by an image capture simulator, or (b) a final simulated sample image merged by the original simulated sample image and the captured sample image (if a merging module is used). Therefore, the simulation model 104 generates a simulated sample image 114.

[0076] The simulated sample images 114 are input as training data into the training algorithm 106. Each simulated sample image 114 within the training data is labeled with a defect type of the defect shown in the simulated sample image 114. Based on the training data, the training algorithm 106 generates an artificial neural network 108. As an example, the training algorithm 106 determines weights associated with connections between artificial neurons of the artificial neural network 108.

[0077] The artificial neural network 108 is applied to one or more captured sample images 115b. It should be noted that captured sample images 115a and captured sample images 115b are both referred to above with reference to Figure 1A The captured sample image 115 is described in detail. However, the captured sample image 115a may be a sample image that includes noise features and / or other features that the user wants to incorporate into the training data of the artificial neural network 108. Conversely, the captured sample image 115b may be a sample image that includes real defects that the user wants to identify and / or analyze. The captured sample image 115a and the captured sample image 115b may be the same or may not be the same.

[0078] The captured sample image 115b shows defects of one or more defect types. The artificial neural network 108 identifies defects in the crystal material as shown in the captured sample image 115b. The artificial neural network 108 determines the defect type of each defect in the crystal material. The user can view the captured sample image 115b to determine whether the artificial neural network's identification of the defect is accurate. For each defect type, a defect recognition rate is determined. The defect recognition rate indicates the accuracy of the artificial neural network in identifying defects of a particular defect type.

[0079] Optionally, a set of defect recognition rates 130 corresponding to various defect types is input into the data model generator 102. The data model generator 102 identifies which of the set of defect recognition rates 130 does not meet a specific standard. As an example, a defect recognition rate below a threshold value does not meet a specific standard. Additionally or alternatively, the lowest defect recognition rate in the defect recognition rate group does not meet a specific standard. The data model generator 102 identifies the defect type corresponding to the defect recognition rate that does not meet the requirements. The identified defect type is a defect type for which the artificial neural network 108 performs poorly. The data model generator 102 generates an additional defective crystal data model 112 associated with the identified defect type. The data model generator 102 avoids generating additional defective crystal data models 112 associated with unidentified defect types.

[0080] The additional defect crystal data model 112 is input into the simulation model 104. The simulation model 104 generates additional simulated sample images 114. The additional simulated sample images 114 show defects of defect types that are poorly represented by the artificial neural network 108. The additional simulated sample images 114 do not show defects of other defect types.

[0081] The additional simulated sample images 114 are input as training data into the training algorithm 106. Based on the additional simulated sample images 114, the training algorithm 106 generates a modified artificial neural network 108.

[0082] Thus, the artificial neural network 108 is trained based on the simulated sample images 114. Furthermore, the performance of the artificial neural network 108 is fed back into the artificial neural network training system to selectively generate additional simulated sample images associated with defect types for which the artificial neural network 108 performs poorly.

[0083] 3. Training artificial neural network using simulated sample images

[0084] Figure 2A B to B illustrate a set of example operations for adapting microscope parameters of an adaptive sample image acquisition system using an artificial neural network in accordance with one or more embodiments. Figure 2AOne or more operations shown in B may be modified, rearranged or omitted altogether. Figure 2A The particular order of operations shown in B to B should not be construed as limiting the scope of one or more embodiments.

[0085] One or more embodiments include obtaining one or more defective crystal data models (operation 202). The one or more defective crystal data models may be specified via user input. As an example, a user may input a set of defective crystal data models through a user interface.

[0086] Additionally or alternatively, one or more defect crystal data models may be generated by modifying previously generated defect crystal data models. As an example, a previously generated defect crystal data model may be associated with a specific crystal material and a specific defect type. Microscopic features and / or noise features of a previously generated defect crystal data model may be modified to generate a series of defect crystal data models. A series of defect crystal data models may be associated with the same crystal material and defect type.

[0087] As an example, a series of defect crystal data models associated with different convergence angles can be generated. The convergence angle can be defined as a half-opening angle, i.e., an angle relative to the optical axis of the microscope. Starting from a specific convergence angle associated with a specific defect crystal data model, the convergence angle can be modified in increments of 1 milliradian (mrad). One defect crystal data model can be associated with a convergence angle of 4 mrad; another defect crystal data model can be associated with a convergence angle of 5 mrad; another defect crystal data model can be associated with a convergence angle of 6 mrad.

[0088] One or more embodiments include applying the defect crystal data model to an image capture simulator to obtain an original simulated sample image (operation 204). The defect crystal data model is input into the image capture simulator. As described above, an example of an image capture simulator is EMsoft. Based on the defect crystal data model, the image capture simulator generates an original simulated sample image.

[0089] One or more embodiments include merging the original simulated sample image with the captured sample image to obtain a final simulated sample image (operation 206). Captured sample images that include noise features and / or other features to be incorporated into the training data for the artificial neural network are identified. As an example, a user may desire that noise generated by dust particles resting on the sample be incorporated into the training data for the artificial neural network. Captured sample images showing dust particles may be identified.

[0090] The captured sample image is merged with the original simulated sample image. Portions of the captured sample image showing desired noise characteristics and / or other characteristics are identified. Portions of the original simulated sample image showing simulated defects are identified. The identified portions of the captured sample image may replace portions of the original simulated sample image not showing simulated defects. The merged sample image may be referred to as a "final simulated sample image" or "simulated sample image". The same captured sample image may be merged with multiple original simulated sample images. Each merge generates a different final simulated sample image.

[0091] As an example, a captured sample image may show a crystalline material including a dust particle. A series of original simulated sample images may show a crystalline material including a simulated defect of a particular defect type. The portion of the captured sample image showing the dust particle may replace a corresponding portion of each original simulated sample image. Thus, a series of final simulated sample images showing simulated defects and dust particles may be generated.

[0092] One or more embodiments include filtering the simulated sample images based on eligibility criteria (operation 208). The eligibility criteria for the simulated sample images are obtained from a data repository. The eligibility criteria are used to determine whether the simulated sample images are qualified as training data for the artificial neural network. As an example, the eligibility criteria may require that the simulated defects in the simulated sample images are associated with a visibility level above a threshold. The visibility level of the simulated defects may be determined based on a contrast level between the simulated defects and surrounding areas of the simulated sample images.

[0093] Each simulated sample image is evaluated based on the eligibility criteria. If the simulated sample image meets the eligibility criteria, the simulated sample image is used as training data. If the simulated sample image does not meet the eligibility criteria, the simulated sample image is not used as training data.

[0094] One or more embodiments include inputting the simulated sample images as training data into a training algorithm to generate an artificial neural network (operation 210). The simulated sample images that meet the qualification criteria are input into the training algorithm. Based on the simulated sample images, the training algorithm determines the weights associated with the connections between artificial neurons within the artificial neural network. Additionally or alternatively, the training algorithm determines other properties of the artificial neural network, such as the connections between artificial neurons, the number of layers of artificial neurons, and / or the functions of artificial neurons.

[0095] One or more embodiments include applying an artificial neural network to obtain identification of machine-generated defects within a set of captured sample images (operation 212). A set of captured sample images is obtained. As an example, the set of captured sample images may be obtained via user input. The captured sample images may include real defects that the user wants to identify and / or analyze.

[0096] The artificial neural network generated at operation 210 is applied to the set of captured sample images. For each captured sample image, the artificial neural network determines whether there are any defects within the captured sample image. If one or more defects are present in the captured sample image, the artificial neural network may determine the location of each defect, the defect type of each defect, and / or other attributes of each defect.

[0097] One or more embodiments include obtaining actual identification of defects within the set of captured sample images (operation 214). A user and / or another application may determine whether there are any actual defects in each captured sample image. The user and / or application may determine the location of each defect, the defect type of each defect, and / or other attributes of each defect. As an example, a user may review each captured sample image to identify any defects within the captured sample image. As another example, a test application may test each sample, which is shown in the captured sample image, to determine whether the sample includes any defects.

[0098] In one embodiment, new training data can be input into the training algorithm at any time to generate a modified artificial neural network. Optionally, captured sample images of actual identifications marked with defects can be fed back into the training algorithm as training data. Thus, the training algorithm can generate a modified artificial neural network.

[0099] One or more embodiments include determining whether there is any correlation between the inaccurate machine-generated defect identification and features of a set of captured sample images (operation 216). For each captured sample image, the machine-generated defect identification determined at operation 212 and the actual defect identification determined at operation 214 are compared to each other.

[0100] If there is a match between the machine-generated defect identification for the particular captured sample image and the actual defect identification for the particular captured sample image, then the machine-generated defect identification for the particular captured sample image is determined to be “accurate.” If there is no match between the machine-generated defect identification for the particular captured sample image and the actual defect identification for the particular captured sample image, then the machine-generated defect identification for the particular captured sample image is determined to be “inaccurate.”

[0101] For each captured sample image, a set of features is determined. The set of features for a particular captured sample image may include any of the following features:

[0102] (a) the defect type of the defect within a particular captured sample image as determined by the artificial neural network at operation 212;

[0103] (b) the defect type of the defect within the particular captured sample image as determined by a user and / or another application at operation 214;

[0104] (c) Features of crystalline materials shown in a particular captured sample image;

[0105] (d) microscope characteristics associated with capturing a particular captured sample image, such as a dwell time or spot size, used to capture the particular captured sample image; and

[0106] (e) Noise characteristics associated with capturing a particular captured sample image, such as whether the particular captured sample image shows any dust particles.

[0107] Determining whether there is any correlation between the inaccurate machine-generated defect identification and the features of the captured sample images. As an example, clustering can be used to determine whether there is any correlation between the inaccurate machine-generated defect identification and the features of the captured sample images. Clustering algorithms include, for example, connectivity-based clustering, centroid-based clustering, distribution-based clustering, and density-based clustering.

[0108] If there is no correlation between the inaccurate machine-generated defect identification and the features of the set of captured sample images, then the artificial neural network may continue to be used without any modification. Operation 212 may be iterated by applying the artificial neural network to additional captured sample images.

[0109] If there is a correlation between the inaccurate machine-generated defect identification and the features of the set of captured sample images, one or more embodiments include identifying one or more features of the captured sample images that are correlated with the inaccurate machine-generated defect identification (operation 218). Features of the captured sample images that are correlated with the inaccurate machine-generated defect identification may be referred to herein as "correlated features."

[0110] As an example, a set of captured sample images can be associated with the following defect identifications and features:

[0111] Accurate / inaccurate defect identification Defect Type Noise characteristics Image 1 Inaccurate Defect Type A Dust particles Image 2 Inaccurate Defect Type A none Image 3 Inaccurate Defect Type A Dust particles Image 4 Accurate Defect Type B Dust particles

[0112] In the above example, each of Image 1, Image 2, and Image 3 is associated with an inaccurate machine-generated defect identification and defect type A. At the same time, Image 1 and Image 3 are associated with dust particles, while Image 2 is not associated with any noise features. Therefore, there may be a correlation between inaccurate defect identification and defect type A. However, there is no correlation between inaccurate defect identification and dust particles. Defect type A can be referred to as a "correlated feature."

[0113] One or more embodiments include obtaining a defect recognition rate for each relevant feature (operation 220). A particular subset of captured sample images associated with a particular relevant feature is identified. A number of accurate machine-generated defect recognitions associated with the particular subset of captured sample images is determined. The number of accurate machine-generated defect recognitions divided by the number of captured sample images within the particular subset is determined as a "defect recognition rate" for the particular relevant feature.

[0114] As an example, inaccurate defect identification may be associated with captured sample images showing defects of defect type A. Defect type A may be referred to as a relevant feature. A subset of captured sample images showing defects of defect type A may include: Image 3, Image 5, Image 19, and Image 23. The defect identification of the subset of captured sample images is as follows:

[0115] Machine generated defect recognition (by ANN) Actual defect identification (by user) Image 3 Defect Type A Defect Type A Image 5 Defect Type B Defect Type A Image 19 Defect Type B Defect Type A Image 23 Defect Type C Defect Type A

[0116] For Image 3, the machine-generated defect identification matches the actual defect identification, so the machine-generated defect identification is determined to be "accurate".

[0117] For each of Image 5, Image 19, and Image 23, the machine-generated defect identification does not match the actual defect identification, and thus the machine-generated defect identification is determined to be "inaccurate".

[0118] Therefore, there is an accurate machine-generated defect identification. A total of four captured sample images are associated with the relevant features of defect type A. Therefore, for the relevant features of defect type A, the defect identification rate is 1 / 4 = 25%.

[0119] As another example, inaccurate defect identification may be associated with captured sample images captured using a convergence angle between 3 and 6 mrad. The convergence angle between 3 and 6 mrad may be referred to as a correlation feature. The convergence angle used to capture the sample image may be as follows:

[0120] Convergence Angle Image 1 3 Image 2 3 Image 3 3 Image 4 7 Image 5 7 Image 6 7 Image 7 6 Image 8 5

[0121] Thus, the subset of captured sample images associated with relevant features may be Image 1, Image 2, Image 3, Image 7, and Image 8.

[0122] The machine-generated defect recognition for images 1, 2, and 3 may be accurate. The machine-generated defect recognition for images 7 and 8 may be inaccurate. Therefore, for the relevant features, the defect recognition rate is 3 / 5 = 60%.

[0123] One or more embodiments include identifying one or more defect recognition rates that match a criterion that needs improvement and a corresponding correlation feature (operation 222). The criterion that needs improvement for the defect recognition rate is obtained from a data repository. The criterion that needs improvement is used to determine areas where the artificial neural network can be improved. In particular, the artificial neural network's recognition of defects shown in certain captured sample images needs improvement. Certain captured sample images are those images associated with correlation features that correspond to defect recognition rates that do not match the criterion that needs improvement.

[0124] Each defect recognition rate corresponding to the corresponding related feature is evaluated based on the criteria that need to be improved. If the defect recognition rate matches the criteria that need to be improved, the related feature corresponding to the defect recognition rate is identified. As further described below with reference to operation 224, an additional defect crystal data model associated with the identified related feature is generated. If the defect recognition rate does not match the criteria that need to be improved, the related feature corresponding to the defect recognition rate is not identified.

[0125] As an example, the criterion for improvement may indicate a threshold. A defect recognition rate below the threshold matches the criterion for improvement. A relevant feature corresponding to the defect recognition rate below the threshold is identified.

[0126] As another example, the defect recognition rates of the related features can be classified and ranked. A certain number of the lowest defect recognition rates can be determined as matching the criteria that need improvement. The related features corresponding to the lowest defect recognition rates are identified.

[0127] One or more embodiments include generating additional defective crystal data models using the identified relevant features (operation 224). A previously generated defective crystal data model is identified. The previously generated defective crystal data model is modified using the relevant features identified at operation 222 to generate a new defective crystal data model associated with the identified relevant features. The new defective crystal data model associated with the identified relevant features may be further modified to generate an additional defective crystal data model associated with the identified relevant features.

[0128] As an example, a previously generated defect crystal data model is associated with the following features: defect type A; xy coordinates of defect location (4, 6); beam current 10 pA; convergence angle 4 mrad. Defect type B may be a relevant feature corresponding to a defect recognition rate that matches the criteria that require improvement.

[0129] A previously generated defect crystal data model may be modified so that defect type A is replaced by defect type B. Thus, a new defect crystal data model may be associated with the following features: defect type B; xy coordinates of defect location (4, 6); beam current 10 pA; convergence angle 4 mrad.

[0130] Additional defect crystal data models can be generated by modifying the new defect crystal data model. The new defect crystal data model can be modified by adjusting the beam current in increments of 0.1 pA. Thus, one defect crystal data model can be associated with the following features: defect type B; xy coordinates of the defect location (4, 6); beam current 10.1 pA; convergence angle 4 mrad. Another defect crystal data model can be associated with the following features: defect type B; xy coordinates of the defect location (4, 6); beam current 10.2 pA; convergence angle 4 mrad.

[0131] As another example, a previously generated defective crystal data model is associated with the following features: defect type A; xy coordinates of defect location (4, 6); beam current 10 pA. Defect type B may be a relevant feature corresponding to a defect recognition rate that matches a criterion that needs improvement.

[0132] A previously generated defect crystal data model may be modified so that defect type B is added to the previously generated defect crystal data model. A new simulated defect of defect type B may be added to the previously generated defect crystal data model. A new defect location may be determined for the simulated defect of defect type B. Thus, the new defect crystal data model may be associated with the following features: defect type A; xy coordinates of defect location (4, 6); defect type B; xy coordinates of defect location (5, 7); beam current 10 pA.

[0133] After generating the additional defective crystal data model, operations 204-210 may be iterated by applying the additional defective crystal data model to the image capture simulator, generating new simulated sample images, and inputting the new simulated sample images as training data into the training algorithm. Thus, the training data may include: previously used simulated sample images and new simulated sample images. Optionally, the training data may include: captured sample images previously analyzed at operation 212, labeled with actual identifications of defects previously obtained at operation 214. Based on the new training data, the training algorithm generates a modified artificial neural network.

[0134] Therefore, the additional training data for the artificial neural network includes simulated sample images associated with features that the artificial neural network previously performed poorly on. Based on the additional training data, the artificial neural network is further trained for specific features that were previously associated with poor performance.

[0135] In other embodiments, operations 214 to 222 are performed. However, both defect crystal data models associated with the identified relevant features (related features associated with the defect recognition rate that matches the standard that needs to be improved) and defect crystal data models not associated with the identified relevant features are generated. But the system focuses on the identified relevant features by generating more defect crystal data models associated with the identified relevant features than defect crystal data models not associated with the identified relevant features. Therefore, the training data used to generate the modified artificial neural network includes simulated sample images associated with a lower defect recognition rate and simulated sample images associated with a higher defect recognition rate.

[0136] In other embodiments, operations 214 to 224 are not necessarily performed. Regardless of the defect recognition rate, an additional defect crystal data model can be generated. Therefore, simulated sample images associated with a lower defect recognition rate and simulated sample images associated with a higher defect recognition rate are generated and input into the training algorithm as training data to generate a modified artificial neural network.

[0137] 4. Example Embodiments

[0138] For the purpose of clarity, detailed examples are described below. The components and / or operations described below should be understood as a specific example that may not be applicable to certain embodiments. Therefore, the components and / or operations described below should not be interpreted as limiting the scope of any claim.

[0139] Figure 3 Examples of simulated sample images for training an artificial neural network are shown in accordance with one or more embodiments.

[0140] The defect crystal data model A includes the following features: defect type A; xy coordinates of the defect position (4, 9), (8, 4), (7, 3); beam current 9.5 pA.

[0141] A simulated sample image 302 is generated based on the defect crystal data model A. As shown, the simulated sample image 302 shows a defect 312 of defect type A.

[0142] The defect crystal data model A is modified to generate the defect crystal data model B. Specifically, the beam current is modified. The defect crystal data model B includes the following features: defect type A; xy coordinates of the defect position (4, 9), (8, 4), (7, 3); beam current 10.5 pA.

[0143] Based on the defect crystal data model B, a simulated sample image 304 is generated. As shown, the simulated sample image 304 shows a defect 314 of defect type A. The defect 314 of the simulated sample image 304 is in the same position as the defect 312 of the simulated sample image 302. However, due to the modified beam current, the defect 314 of the simulated sample image 304 looks slightly different from the defect 312 of the simulated sample image 302.

[0144] Additionally, the defect crystal data model C includes the following features: defect type B; xy coordinates of the defect position (3, 9), (9, 2); beam current 9.5 pA.

[0145] Based on the defect crystal data model C, a simulated sample image 306 is generated. As shown, the simulated sample image 306 shows a defect 316 of defect type B. The defect 316 of the simulated sample image 306 looks different from the defect 312 and the defect 314 because the defect 316 belongs to defect type B, while the defect 312 and the defect 314 belong to defect type A.

[0146] The defect crystal data model C is modified to generate a defect crystal data model D. Specifically, the beam current is modified. The defect crystal data model D includes the following features: defect type B; xy coordinates of the defect position (3, 9), (9, 2); beam current 10.5 pA.

[0147] Based on the defect crystal data model D, a simulated sample image 308 is generated. As shown, the simulated sample image 308 shows a defect 318 of defect type B. The defect 318 of the simulated sample image 308 is in the same position as the defect 316 of the simulated sample image 306. However, due to the modified beam current, the defect 318 of the simulated sample image 308 looks slightly different from the defect 316 of the simulated sample image 306.

[0148] The simulated sample images 302-308 are generated based on the defect crystal data model without using any image data and without using any microscope to capture sample images of real samples. The simulated sample images 302-308 are input into the training algorithm as training data. The training algorithm generates an artificial neural network that is configured to recognize defects in the crystal material as shown in the captured sample images.

[0149] 5. Hardware Overview

[0150] According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. Special-purpose computing devices may be hardwired to perform these techniques, or may include digital electronic devices, such as one or more application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or network processing units (NPUs), which are permanently programmed to perform these techniques, or may include one or more general-purpose hardware processors, which are programmed to perform these techniques according to program instructions in firmware, memory, other storage bodies, or a combination thereof. Such special-purpose computing devices may also combine customized hardwired logic, ASICs, FPGAs, or NPUs with custom programming to implement these techniques. Special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, network devices, or any other devices that implement these techniques in combination with hardwiring and / or program logic.

[0151] For example, Figure 4 4 is a block diagram illustrating a computer system 400 upon which embodiments of the present invention may be implemented. Computer system 400 includes a bus 402 or other communication mechanism for communicating information, and a hardware processor 404 coupled to bus 402 for processing information. Hardware processor 404 may be, for example, a general purpose microprocessor.

[0152] The computer system 400 also includes a main memory 406, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 402 for storing information and instructions to be executed by the processor 404. The main memory 406 may also be used for storing temporary variables or other intermediate information during execution of instructions by the processor 404. When such instructions are stored in a non-transitory storage medium accessible to the processor 404, the computer system 400 is presented as a special-purpose machine customized to perform the operations specified in the instructions.

[0153] Computer system 400 further includes a read only memory (ROM) 408 or other static storage device coupled to bus 402 for storing static information and instructions for processor 404. A storage device 410, such as a magnetic or optical disk, is provided and coupled to bus 402 for storing information and instructions.

[0154] The computer system 400 may be coupled via the bus 402 to a display 412, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 414, including alphanumeric and other keys, is coupled to the bus 402 for communicating information and command selections to the processor 404. Another type of user input device is a cursor control 416, such as a mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processor 404, and for controlling cursor movement on the display 412. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), which allows the device to specify a position in a plane.

[0155] Computer system 400 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, makes or programs computer system 400 into a special purpose machine. According to one embodiment, the techniques herein are performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another storage medium such as storage device 410. Execution of the sequences of instructions contained in main memory 406 causes processor 404 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0156] As used herein, the term "storage medium" refers to any non-temporary medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 410. Volatile media include dynamic memory, such as main memory 406. Common forms of storage media include, for example, floppy disks, floppy disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical medium with a hole pattern, RAM, PROM and EPROM, Flash-EPROM, NVRAM, any other memory chip or cartridge, content addressable memory (CAM) and ternary content addressable memory (TCAM).

[0157] Storage media are distinct from transmission media but can be used in conjunction with transmission media. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including the wires that make up bus 402. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0158] Various forms of media may be involved when one or more sequences of one or more instructions are carried to processor 404 for execution. For example, the instructions may initially be carried to a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. The modem local to computer system 400 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on bus 402. Bus 402 carries the data to main memory 406, from which processor 404 retrieves and executes the instructions. The instructions received by main memory 406 may optionally be stored on storage device 410 before or after processor 404 executes.

[0159] Computer system 400 also includes a communication interface 418 coupled to bus 402. Communication interface 418 provides bidirectional data communication coupled to network link 420, which is connected to local network 422. For example, communication interface 418 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 418 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN. A wireless link can also be implemented. In any such implementation, communication interface 418 sends and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0160] The network link 420 typically provides data communication to other data devices through one or more networks. For example, the network link 420 can provide a connection to a host computer 424 or to data equipment operated by an Internet Service Provider (ISP) 426 through a local network 422. The ISP 426 in turn provides data communication services through the global packet data communication network now commonly referred to as the "Internet" 428. Both the local network 422 and the Internet 428 use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 420 and through the communication interface 418 (which carry the digital data to and from the computer system 400) are example forms of transmission media.

[0161] Computer system 400 can send messages and receive data, including program code, through the network, network link 420 and communication interface 418. In the Internet example, server 430 can transmit the requested code for an application program through Internet 428, ISP 426, local network 422 and communication interface 418.

[0162] The received code may be executed by processor 404 as it is received, and / or stored in storage device 410 or other non-volatile storage for later execution.

[0163] 6. Miscellaneous; Extension

[0164] An embodiment provides a system having one or more devices, the devices comprising a hardware processor and configured to perform any of the operations described herein and / or recited in any of the claims below.

[0165] In an embodiment, a non-transitory computer-readable storage medium includes instructions that, when executed by one or more hardware processors, cause performance of any of the operations described herein and / or recited in any of the claims.

[0166] According to one or more embodiments, any combination of the features and functions described herein may be used. In the foregoing description, embodiments have been described with reference to many specific details, which may vary depending on the implementation. Therefore, the description and drawings are to be considered illustrative rather than restrictive. The sole and exclusive indication of the scope of the invention, and what the applicant intends as an indication of the scope of the invention, is the literal and equivalent scope of the set of claims issued from this application in the specific form in which these claims are issued, including any subsequent amendments.

Claims

1. A non-transitory computer-readable medium comprising instructions that, when executed by one or more hardware devices, cause operations to be performed, the operations comprising: Acquire a first data model comprising: (a) one or more characteristics of a crystalline material, (b) one or more characteristics of a first defect type, and (c) one or more characteristics of a first microscope; applying at least the first data model to a simulation model to generate a first simulated sample image, wherein the first simulated sample image shows the crystalline material including a first set of one or more defects of the first defect type; obtaining a second data model comprising: (a) the one or more characteristics of the crystalline material, (b) the one or more characteristics of the first defect type, and (c) one or more characteristics of a second microscope different from the one or more characteristics of the first microscope; applying at least the second data model to the simulation model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a second set of one or more defects of the first defect type; inputting a set of one or more simulated sample images including the first simulated sample image and the second simulated sample image as training data into a training algorithm to generate an artificial neural network; The artificial neural network is generated, wherein the artificial neural network is configured to detect a third set of one or more defects of the first defect type in the crystalline material shown in a set of one or more scanned sample images.

2. The medium of claim 1 , wherein the one or more characteristics of the crystalline material include one or more of: (a) one or more characteristics of a substrate of the crystalline material, and (b) one or more characteristics of a target layer of the crystalline material located above the substrate.

3. The medium according to claim 1, wherein The first microscope comprises an electron microscope. 4 . The medium of claim 1 , wherein obtaining the second data model comprises generating the second data model by modifying the first data model.

5. The medium of claim 1, wherein: The one or more characteristics of the first microscope include a first value of a microscope parameter of the first microscope; The one or more characteristics of the second microscope include a second value of the microscope parameter of the second microscope; and The microscope parameters include at least one of the following: Duration of stay; Resolution; Accelerating voltage; beam flow; Convergence angle; and Spot size.

6. The medium according to claim 1, wherein The first data model further includes: one or more characteristics of the first image noise.

7. The medium of claim 6, wherein: The operations further include: acquiring a second data model comprising: (a) the one or more characteristics of the crystalline material, (b) the one or more characteristics of the first defect type, and (c) one or more characteristics of a second image noise different from the one or more characteristics of the first image noise; applying at least the second data model to the simulation model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a third set of one or more defects of the first defect type; The simulated sample image group includes: the first simulated sample image and the second simulated sample image.

8. The medium of claim 1, wherein applying the first data model to the simulation model to generate the first simulated sample image comprises: applying the first data model to a scanned simulation model to generate a raw simulated sample image showing the crystalline material including the first set of defects of the first defect type; acquiring a scanned sample image showing the crystalline material; combining (a) a portion of the original simulated sample image and (b) a portion of the scanned sample image; The first simulated sample image is generated including (a) the portion of the original simulated sample image and (b) the portion of the scanned sample image.

9. The medium according to claim 8, wherein The portion of the scanned sample image includes dust particles.

10. The medium according to claim 1, wherein The operations further include: obtaining a second data model comprising: (a) the one or more characteristics of the crystalline material, and (b) the one or more characteristics of the first defect type; applying the second data model to the simulated model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a third set of one or more defects of the first defect type; determining whether visibility of the third set of one or more defects meets a criterion; In response to determining that the visibility of the third set of one or more defects does not meet a criterion: refraining from inputting the second simulated sample image as the training data into the training algorithm to generate the artificial neural network.

11. The medium according to claim 1, wherein In response to determining that the visibility of the first set of one or more defects meets a criterion, the set of simulated sample images including the first simulated sample image is input as the training data into the training algorithm to generate the artificial neural network.

12. The medium of claim 1, wherein: The operations further include: obtaining a second data model comprising: (a) the one or more characteristics of the crystalline material, and (b) one or more characteristics of a second defect type; applying at least the second data model to the simulation model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a third set of one or more defects of the second defect type; The simulated sample image group includes: the first simulated sample image and the second simulated sample image; The artificial neural network is further configured to detect a fourth set of one or more defects of the second defect type in the crystalline material shown in the set of scanned sample images.

13. The medium according to claim 12, wherein: The operations further include: obtaining information indicating a first accuracy of the artificial neural network for detecting the second set of defects of the first defect type and a second accuracy of the artificial neural network for detecting a fourth set of defects of the second defect type; In response to determining that the first accuracy is less than the second accuracy; obtaining a third data model comprising: (a) the one or more characteristics of the crystalline material, and (b) the one or more characteristics of the first defect type; applying a third data set to the simulation model to generate a third simulated sample image, wherein the third simulated sample image shows the crystalline material including a fifth set of one or more defects of the first defect type; inputting a second set of simulated sample images as the training data into the training algorithm to generate a modified artificial neural network, wherein the second set of simulated sample images (a) includes the set of simulated sample images and the third simulated sample image, and (b) does not include any additional simulated sample images of the crystalline material showing any defects including the second defect type; The modified artificial neural network is generated.

14. The medium of claim 1, applying said artificial neural network to detect said second set of defects of said first defect type in said crystalline material shown in said set of scanned sample images.

15. A method for sample image analysis, comprising: Acquire a first data model comprising: (a) one or more characteristics of a crystalline material, (b) one or more characteristics of a first defect type, and (c) one or more characteristics of a first microscope; applying the first data model to a simulation model to generate a first simulated sample image, wherein the first simulated sample image shows the crystalline material including a first set of one or more defects of the first defect type; obtaining a second data model comprising: (a) the one or more characteristics of the crystalline material, (b) the one or more characteristics of the first defect type, and (c) one or more characteristics of a second microscope different from the one or more characteristics of the first microscope; applying at least the second data model to the simulation model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a second set of one or more defects of the first defect type; inputting a set of one or more simulated sample images including the first simulated sample image and the second simulated sample image as training data into a training algorithm to generate an artificial neural network; generating the artificial neural network, wherein the artificial neural network is configured to detect a third set of one or more defects of the first defect type in the crystalline material shown in a set of one or more scanned sample images; The method is performed using at least one device including a hardware processor.

16. The method of claim 15, wherein the one or more features of the crystalline material include one or more of: (a) one or more features of a substrate of the crystalline material, and (b) one or more features of a target layer of the crystalline material located above the substrate.

17. The method of claim 15, wherein the first data model further comprises: One or more features of a first microscope.

18. A system for sample image analysis, comprising: at least one device comprising a hardware processor; as well as The system is configured to perform operations including: Acquire a first data model comprising: (a) one or more characteristics of a crystalline material, (b) one or more characteristics of a first defect type, and (c) one or more characteristics of a first microscope; applying the first data model to a simulation model to generate a first simulated sample image, wherein the first simulated sample image shows the crystalline material including a first set of one or more defects of the first defect type; obtaining a second data model comprising: (a) the one or more characteristics of the crystalline material, (b) the one or more characteristics of the first defect type, and (c) one or more characteristics of a second microscope different from the one or more characteristics of the first microscope; applying at least the second data model to the simulation model to generate a second simulated sample image, wherein the second simulated sample image shows the crystalline material including a second set of one or more defects of the first defect type; inputting a set of one or more simulated sample images including the first simulated sample image and the second simulated sample image as training data into a training algorithm to generate an artificial neural network; The artificial neural network is generated, wherein the artificial neural network is configured to detect a third set of one or more defects of the first defect type in the crystalline material shown in a set of one or more scanned sample images.

Images