Substrate Defect Analysis

Abstract

The method includes identifying characteristic data for a substrate processed by a substrate processing system. The method further includes identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data. The method further includes subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data. The method further includes triggering execution of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories.

Description

[Technical Field]

[0001] The present disclosure relates to defect analysis, and more particularly to substrate (e.g., wafer) defect analysis and root cause analysis. [Background technology]

[0002] Production equipment is used to manufacture products (e.g., substrates). For example, semiconductor substrate processing equipment is used to manufacture semiconductor substrates (e.g., semiconductor substrates having integrated circuits thereon). Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to delineate any scope of particular embodiments of the disclosure or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes identifying characteristic data for a substrate processed by a substrate processing system. The method further includes identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data. The method further includes subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data. The method further includes triggering execution of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories.

[0005] In another aspect of the present disclosure, a non-transitory computer-readable storage medium stores instructions that, when executed, cause a processing device to perform operations. The operations include identifying characteristic data for a substrate processed by a substrate processing system. The operations further include identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data. The operations further include subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data. The operations further include triggering execution of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories.

[0006] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device is for identifying characteristic data of a substrate processed by a substrate processing system. The processing device is further for identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data. The processing device is further for subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data. The processing device is further for triggering execution of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories.

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system configuration, according to certain embodiments. [Figure 2] FIG. 1 illustrates a dataset generator for creating a dataset for a machine learning model, according to certain embodiments. [Figure 3] FIG. 1 is a block diagram illustrating determining predictive data, according to certain embodiments. [Figure 4A] FIG. 1 is a block diagram illustrating defect categories, in accordance with certain embodiments. [Figure 4B] FIG. 10 is a block diagram illustrating defect subcategories, in accordance with certain embodiments. [Figure 4C] FIG. 10 is a block diagram associated with defect subcategories, according to certain embodiments. [Figure 4D] FIG. 1 is a block diagram illustrating substrate defects, in accordance with certain embodiments. [Figures 5A-5C] 1 is a flowchart of a method relating to defect analysis, in accordance with certain embodiments. [Figure 6] FIG. 1 is a block diagram illustrating a computer system, in accordance with certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Described herein are techniques for substrate defect analysis (eg, defect source tracking and defect root cause identification and / or corrective action recommendations).

[0010] Production equipment uses production parameters to manufacture products. For example, substrate processing equipment uses production parameters (e.g., temperature, pressure, etc.) during substrate processing operations (e.g., layer deposition, etching, etc.) to manufacture substrates. As a result of one or more operations, a substrate (e.g., a completed substrate, a partially processed substrate) may have anomalies (e.g., defects). A substrate with anomalies may have performance data that does not meet a threshold value (e.g., is a bad wafer). This can result in discarded substrates, poorly performing substrates, low yields, wasted material and energy, etc.

[0011] Traditionally, actual or test substrates are processed by substrate processing equipment and then manually inspected to identify defects and attempt to determine the source and root cause of the defects in order to reduce material exposure and improve equipment repair and mean time to recovery. Manual inspection is time-consuming, can be inaccurate, is user-dependent, causes damage to substrate processing equipment, and uses more energy and materials. Manual attempts to determine defect sources, root causes, and corrective actions associated with the defects are time-consuming and inaccurate. This can result in reduced throughput, interrupted production, production of substrates with performance data that does not meet thresholds, etc.

[0012] The devices, systems, and methods disclosed herein provide substrate defect analysis (e.g., defect source tracking and defect root cause identification, including corrective action recommendations to improve mean time to equipment repair).

[0013] The processing device identifies characteristic data for substrates processed by the substrate processing system, hi some embodiments, the characteristic data is metrology data received from a metrology instrument.

[0014] The processing device identifies regions of the substrate corresponding to a first defect category based on the first subset of characteristic data. In some examples, the first subset of characteristic data includes scanning electron microscope (SEM) images, energy dispersive X-ray microanalysis (EDX) images, etc. In some examples, the first defect category includes line breaks, non-fills, bridges, stains, scratches, glass damage, foreign matter (e.g., particles), residues, resist collapse, via stress, voids, pits, crystalline defects, cracks, etc.

[0015] The processing device subdivides regions of the substrate corresponding to the first defect category into defect subcategories based on a second subset of the characteristic data. In some examples, the second subset of the characteristic data includes morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc. In some examples, the defect subcategories include spherical particles, random particles, rod-shaped particles, flake particles, post-imprint fall-on particles, ring pits, micropits, macropits, mouse bites, chemical mechanical polishing (CMP) bridging, photoresist bridging, microbridging, organic stains, inorganic stains, lithography pitting, step bunching, stacking faults, shallow triangles, obtuse triangles, surface triangles, downfalls, tics, chatter marks, crescents, micropipes, photoluminescence (PL) circles, basal plane dislocations, protrusions, hillocks, etc.

[0016] The processing device triggers the execution of a corrective action associated with the substrate processing system based on at least one of the plurality of defect subcategories. In some embodiments, the corrective action includes issuing an alert, triggering a cleaning process, triggering a repair process, replacing substrate processing equipment parts, triggering further inspection, triggering computational process control (CPC), statistical process control (SPC) (e.g., SPC comparing to a three-sigma graph, etc.), advanced process control (APC), model-based process control, triggering preventative maintenance, triggering design optimization, updating production parameters, triggering wafer recipe modifications, triggering feedback control, triggering machine learning modifications, etc.

[0017] Aspects of the present disclosure provide technical advantages. The present disclosure avoids the time, inaccuracy, and subjectivity of conventional systems. The present disclosure produces substrates with characteristic data that better meet thresholds, reduces damage to substrate processing equipment, increases throughput, reduces production interruptions, reduces the use of test wafers, etc., compared to conventional solutions.

[0018] Although some embodiments of the present disclosure refer to substrate processing equipment and defects in substrates, in some embodiments, the present disclosure may be applied to other types of production equipment, other types of products, and other types of anomalies.

[0019] FIG. 1 is a block diagram illustrating an example system 100 (an example system configuration) according to certain embodiments. System 100 (e.g., corrective action component 122 and / or prediction component 114) can perform methods described herein (e.g., methods 500A-500C of FIGS. 5A-5C). System 100 includes client devices 120, production equipment 124, sensors 126, measurement equipment 128, prediction server 112, and data store 140. In some embodiments, prediction server 112 is part of prediction system 110. In some embodiments, prediction system 110 further includes server machines 170 and 180.

[0020] In some embodiments, one or more of client device 120, production equipment 124, sensors 126, measurement equipment 128, prediction server 112, data store 140, server machine 170, and / or server machine 180 are coupled to one another via network 130 to generate prediction data 160 for performing defect source tracking and defect root cause identification. In some embodiments, network 130 is a public network that provides client device 120 with access to prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 with access to production equipment 124, sensors 126, measurement equipment 128, data store 140, and other privately available computing devices. In some embodiments, network 130 includes one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0021] In some embodiments, client device 120 includes a computing device such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet computer, a netbook computer, or the like. In some embodiments, client device 120 includes a corrective action component 122. In some embodiments, corrective action component 122 may also be included in prediction system 110 (e.g., a machine learning processing system). In some embodiments, corrective action component 122 is alternatively included in prediction system 110 (e.g., instead of being included in client device 120). Client device 120 includes an operating system that enables a user to one or more of: integrate, generate, review, or edit data, and provide instructions to prediction system 110 (e.g., a machine learning processing system).

[0022] In some embodiments, the corrective action component 122 receives one or more of user input (e.g., via a graphical user interface (GUI) displayed via the client device 120), characteristic data 142, performance data 152, etc. In some embodiments, the corrective action component 122 transmits data (e.g., user input, characteristic data 142, performance data 152, etc.) to the prediction system 110, receives prediction data 160 from the prediction system 110, determines corrective actions based on the prediction data 160, and causes the corrective actions to be implemented. In some embodiments, the corrective action component 122 stores the data (e.g., user input, characteristic data 142, performance data 152, etc.) in the data store 140, and the prediction server 112 retrieves the data from the data store 140. In some embodiments, the prediction server 112 stores the output of the trained machine learning model 190 (e.g., prediction data 160) in the data store 140, and the client device 120 retrieves the output from the data store 140. In some embodiments, the corrective action component 122 receives instructions for corrective actions (eg, based on the predictive data 160) from the predictive system 110 and causes the corrective actions to be carried out.

[0023] In some embodiments, the predictive data 160 is associated with a corrective action. In some embodiments, the corrective action is associated with one or more of cleaning one or more pieces of production equipment 124 (e.g., processing chambers), repairing one or more pieces of production equipment 124, replacing one or more pieces of production equipment 124, computational process control (CPC), statistical process control (SPC) (e.g., SPC comparing to a three-sigma graph, etc.), advanced process control (APC), model-based process control, preventive operational maintenance, design optimization, production parameter updates, wafer recipe modifications, feedback control, machine learning modifications, etc. In some embodiments, the corrective action includes providing a warning (e.g., an alert to not use one or more pieces of production equipment 124 if the predictive data 160 indicates a predicted anomaly). In some embodiments, the corrective action includes performing feedback control (e.g., cleaning, repairing, and / or replacing one or more pieces of production equipment 124 in response to the predictive data 160 indicating a predicted anomaly). In some embodiments, the corrective action includes providing machine learning (e.g., causing modification of one or more pieces of production equipment 124 based on predictive data 160).

[0024] In some embodiments, prediction server 112, server machine 170, and server machine 180 each include a computing device such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), or the like.

[0025] The prediction server 112 includes a prediction component 114. In some embodiments, the prediction component 114 receives substrate characteristic data 142 (e.g., received from the client device 120 and retrieved from the data store 140) and generates prediction data 160 related to the execution of corrective actions (e.g., defect analysis, defect source tracking, defect root cause identification, defect translation, defect evolution, etc.). In some embodiments, the prediction component 114 determines the prediction data 160 using one or more trained machine learning models 190. In some embodiments, the trained machine learning models 190 are trained using historical characteristic data 144 and historical performance data 154.

[0026] In some embodiments, the prediction system 110 (e.g., prediction server 112, prediction component 114) uses supervised machine learning (e.g., supervised dataset, historical characteristic data 144 labeled with historical performance data 154, etc.) to generate the predicted data 160. In some embodiments, the prediction system 110 uses semi-supervised learning (e.g., semi-supervised dataset, performance data 152 is a predictive percentage, etc.) to generate the predicted data 160. In some embodiments, the prediction system 110 uses unsupervised machine learning (e.g., unsupervised dataset, clustering, clustering based on historical characteristic data 144, etc.) to generate the predicted data 160.

[0027] In some embodiments, the production equipment 124 (e.g., cluster tools, wafer backgrinding systems, wafer saw equipment, die attach machines, wire bonders, die overcoat systems, molding equipment, hermetic sealing equipment, metal can welding machines, DTFS machines, branding equipment, lead finishing equipment, etc.) is part of a substrate processing system (e.g., an integrated processing system). The production equipment 124 includes one or more of a controller, an enclosure system (e.g., substrate carriers, front opening unified pods (FOUPs), autoteach FOUPs, process kit enclosure systems, substrate enclosure systems, cassettes, etc.), a side storage pod (SSP), an aligner device (e.g., aligner chambers), a factory interface (e.g., a front end of equipment module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robotic arm (e.g., disposed in a transfer chamber, disposed in a front interface, etc.), etc. The enclosure system, SSP, and load locks are attached to a factory interface, and a robot arm disposed at the factory interface can transfer content (e.g., substrates, process kit rings, carriers, validation wafers, etc.) between the enclosure system, SSP, load locks, and factory interface. An aligner device is disposed at the factory interface to align the content. The load locks and processing chambers are attached to a transfer chamber, and a robot arm disposed at the transfer chamber can transfer content (e.g., substrates, process kit rings, carriers, validation wafers, etc.) between the load locks, processing chambers, and transfer chambers. In some embodiments, the production equipment 124 includes components of a substrate processing system. In some embodiments, the substrate characteristic data 142 results from a substrate undergoing one or more processes (e.g., etching, heating, cooling, transferring, treating, flowing, etc.) performed by components of the production equipment 124.

[0028] In some embodiments, the sensors 126 provide characteristic data 142 (e.g., sensor values, such as historical and current sensor values) of substrates processed by the production equipment 124. In some embodiments, the sensors 126 include one or more of an imaging sensor (e.g., an SEM, a camera, an imaging device, etc.), a pressure sensor, a temperature sensor, a flow sensor, a spectroscopic sensor, etc. In some embodiments, the characteristic data 142 is used for equipment health and / or product health (e.g., product quality). In some embodiments, the characteristic data 142 is received over a period of time.

[0029] In some embodiments, the sensor 126 and / or metrology equipment 128 provides characteristic data 142 including one or more of image data, SEM images, EDX images, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, or defect metadata including, but not limited to, gray level data and signal-to-noise data, temperature, spacing, current, power, voltage, etc.

[0030] In some embodiments, the characteristic data includes SEM images (e.g., images captured by a scanning electron microscope, which uses a focused electron beam to scan the surface of the substrate to create a high-resolution image). In some embodiments, the characteristic data includes EDX images (e.g., images generated from data collected using X-ray techniques to identify the elemental composition of a material). In some embodiments, the characteristic data includes morphology data (e.g., data related to the morphology of the substrate). In some embodiments, the characteristic data includes size attribute data (e.g., data describing the size of an attribute of the substrate). In some embodiments, the characteristic data includes dimensional attribute data (e.g., data describing the dimensions of an attribute of the substrate). In some embodiments, the characteristic data includes defect distribution data (e.g., data describing the distribution (e.g., spatial, temporal, etc.) of defects in the substrate). In some embodiments, the characteristic data includes spatial location data (e.g., data describing the spatial location of an attribute, defect, element, etc., in the substrate). In some embodiments, the characteristic data includes elemental analysis data (e.g., data describing the elemental composition of the substrate). In some embodiments, the characteristic data includes wafer signature data (e.g., data describing the distribution of wafer defects on a substrate resulting from a single production issue). In some embodiments, the characteristic data includes chip layer data (e.g., a particular layer or operation in a substrate production process). In some embodiments, the characteristic data includes chip layout data (e.g., data describing the layout of a substrate in terms of planar geometry). In some embodiments, the characteristic data includes edge data (e.g., data describing an edge of a wafer (e.g., chipped edge, wafer edge thickness, wafer bow, and / or wafer warp)). In some embodiments, the characteristic data includes defect metadata including, but not limited to, gray level data (e.g., data describing the brightness of a pixel in an image of the substrate) and signal-to-noise data (e.g., data describing the signal-to-noise ratio of a substrate measurement, for example, by a spectroscopic measurement instrument).

[0031] In some embodiments, characteristic data 142 (e.g., historical characteristic data 144, current characteristic data 146, etc.) is processed (e.g., by client device 120 and / or by prediction server 112). In some embodiments, processing characteristic data 142 includes generating features. In some embodiments, the features are patterns in characteristic data 142 (e.g., slope, width, height, peaks, etc.) or combinations of values ​​from characteristic data 142 (e.g., power derived from voltage and current, etc.). In some embodiments, characteristic data 142 includes features used by prediction component 114 to derive prediction data 160.

[0032] In some embodiments, metrology equipment 128 (e.g., imaging equipment, spectroscopic equipment, ellipsometry equipment, etc.) is used to determine metrology data (e.g., inspection data, image data, spectroscopic data, ellipsometry data, material composition, optical, or structural data, etc.) corresponding to substrates produced by production equipment 124 (e.g., substrate processing equipment). In some examples, metrology equipment 128 is used to inspect a portion (e.g., a layer) of a substrate after production equipment 124 processes the substrate. In some embodiments, metrology equipment 128 performs scanning acoustic microscopy (SAM), ultrasound inspection, X-ray inspection, and / or computed tomography (CT) inspection. In some examples, metrology equipment 128 is used to determine the quality of the processed substrate (e.g., layer thickness, layer uniformity, interlayer spacing, etc.) after production equipment 124 deposits one or more layers on the substrate. In some embodiments, metrology equipment 128 includes an imaging device (e.g., SAM equipment, ultrasound equipment, X-ray equipment, CT equipment, etc.). In some embodiments, characteristic data 142 includes sensor data from sensors 126 and / or measurement data from measurement devices 128. In some embodiments, performance data 152 includes user input via client device 120 and / or measurement data from measurement devices 128. Characteristic data 142 can include measurement data from a first subset of measurement devices 128, and performance data 152 can include measurement data from a second subset of measurement devices 128.

[0033] In some embodiments, data store 140 is memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. In some embodiments, data store 140 includes multiple storage components (e.g., multiple drives or multiple databases) across multiple computing devices (e.g., multiple server computers). In some embodiments, data store 140 stores one or more of characteristic data 142, performance data 152, and / or prediction data 160.

[0034] Characterization data 142 includes historical characterization data 144 and current characterization data 146. In some embodiments, characterization data 142 can include one or more of image data, SEM images, EDX images, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc. In some embodiments, sensor data can include temperature data, temperature range, power data, comparison parameters for comparing inspection data to threshold data, threshold data, cooling rate data, cooling rate range, etc. In some embodiments, at least a portion of characterization data 142 is from sensors 126 and / or metrology instruments 128.

[0035] Performance data 152 includes historical performance data 154 and current performance data 156. Performance data 152 may indicate whether a substrate is properly designed, properly manufactured, and / or properly functioning. In some embodiments, at least a portion of performance data 152 relates to the quality of substrates produced by production equipment 124. In some embodiments, at least a portion of performance data 152 is based on metrology data from metrology equipment 128 (e.g., historical performance data 154 includes metrology data indicative of properly processed substrates, substrate characteristic data, yield, etc.). In some embodiments, at least a portion of performance data 152 is based on inspection of the substrates (e.g., current performance data 156 based on actual inspection). In some embodiments, performance data 152 includes user input (e.g., via client device 120) indicative of the quality of the substrates. In some embodiments, the performance data 152 includes an indication of an absolute value (e.g., a deformation value that falls short of the threshold deformation value by a calculated value, indicating that the bond interface inspection data is below the threshold data by a calculated value) or a relative value (e.g., a deformation that falls short of the threshold deformation by 5%, indicating that the bond interface inspection data is below the threshold data by 5%). In some embodiments, the performance data 152 indicates that a threshold amount of error (e.g., at least 5% error in manufacturing, at least 5% error in flow rate, at least 5% error in deformation, specification limits) has been met.

[0036] In some embodiments, client device 120 provides performance data 152 (e.g., product data). In some examples, client device 120 provides (e.g., based on user input) performance data 152 indicating product anomalies (e.g., defective products). In some embodiments, performance data 152 includes the amount of product manufactured that was normal or abnormal (e.g., 98% normal product). In some embodiments, performance data 152 indicates the amount of product being manufactured that is predicted to be normal or abnormal. In some embodiments, performance data 152 includes one or more of the yield of a previous batch of product, the average yield, the predicted yield, the predicted amount of defective or non-defective product, etc. In some examples, in response to a first batch of product having a yield of 98% (e.g., 98% of the product were normal and 2% were abnormal), client device 120 provides performance data 152 indicating that an upcoming batch of product may have a 98% yield.

[0037] In some embodiments, the historical data includes one or more of historical characteristic data 144 and / or historical performance data 154 (e.g., at least a portion for training machine learning model 190). The current data includes one or more of current characteristic data 146 and / or current performance data 156 (e.g., at least a portion that is input to trained machine learning model 190 after training model 190 using historical data). In some embodiments, the current data is used to retrain trained machine learning model 190.

[0038] In some embodiments, the predictive data 160 may be used to trigger the implementation of corrective actions on production equipment, substrate processing systems, or substrate processing equipment components.

[0039] Performing multiple types of measurements on multiple layers of a product to determine whether corrective action should be taken is costly in terms of time used, metrology equipment 128 used, energy consumed, bandwidth used to send the metrology data, processor overhead to process the metrology data, etc. By providing sensor data 142 to the model 190 and receiving predicted data 160 from the model 190, the system 100 has the technical advantage of using multiple types of metrology equipment 128 on multiple layer products and avoiding the costly process of scrapping substrates.

[0040] Running a production process using production equipment 124 and / or production parameters that result in defective product is costly in terms of time, energy, product, production equipment 124, cost of identifying corrective actions to avoid resulting in defective product, etc. By providing characteristic data 142 to model 190 and receiving prediction data 160 from model 190, system 100 has the technical advantage of using metrology equipment 128 and avoiding the costly process of scrapping substrates.

[0041] In some embodiments, the prediction system 110 further includes a server machine 170 and a server machine 180. The server machine 170 includes a dataset generator 172 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing the machine learning model 190. The dataset generator 172 has the functionality to collect, compile, reduce, and / or partition data to prepare the data for machine learning. In some embodiments (e.g., for small datasets), partitioning for training and validation (e.g., explicit partitioning) is not used. Repeated cross-validation (e.g., 5-fold cross-validation, leave-one-out cross-validation) may be used during training, in which a given dataset is actually repeatedly partitioned into different training and validation sets during training. A model (e.g., the best model, the model with the highest accuracy, etc.) is automatically selected from the vector of models on the separated combined subset. In some embodiments, the dataset generator 172 may explicitly divide the historical data (e.g., historical characteristic data 144 and corresponding historical performance data 154) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In this embodiment, some operations of the dataset generator 172 are described in detail below with respect to FIG. 2. In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of features (e.g., training features). In some examples, the first set of features corresponds to a first set of types of characteristic data (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern in values ​​from the first set of sensors) corresponding to each of the datasets (e.g., a training set, a validation set, and a test set), and the second set of features corresponds to a second set of types of characteristic data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the datasets.

[0042] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. In some embodiments, engines (e.g., training engine 182, validation engine 184, selection engine 185, and test engine 186) refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 can train machine learning model 190 using one or more sets of features associated with a training set from dataset generator 172. In some embodiments, training engine 182 generates multiple trained machine learning models 190, each corresponding to a distinct set of parameters (e.g., characteristic data 142) and corresponding responses (e.g., performance data 152) of the training set. In some embodiments, multiple models are trained with the same parameters with distinct targets for the purpose of modeling multiple effects. In some examples, a first trained machine learning model was trained using feature data 142 from all sensors 126 (e.g., sensors 1-5), a second trained machine learning model was trained using a first subset of feature data (e.g., from sensors 1, 2, and 4), and a third trained machine learning model was trained using a second subset of feature data (e.g., from sensors 1, 3, 4, and 5) that partially overlaps with the first subset of features.

[0043] The validation engine 184 may validate the trained machine learning models 190 using a corresponding set of validation set features from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of training set features is validated using a first set of validation set features. The validation engine 184 determines the accuracy of each of the trained machine learning models 190 based on the corresponding set of validation set features. The validation engine 184 evaluates and flags (e.g., discards) trained machine learning models 190 that have an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may select one or more trained machine learning models 190 that have an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may select the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190.

[0044] The test engine 186 can test the trained machine learning models 190 using a corresponding set of test set features from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of training set features is tested using a first set of test set features. The test engine 186 determines the trained machine learning model 190 with the highest accuracy among all of the trained machine learning models based on the test set.

[0045] In some embodiments, machine learning model 190 (e.g., used for classification) refers to a model artifact created by training engine 182 using a training set that includes data inputs and corresponding target outputs (e.g., correctly classifying a condition or ordinal level for each training input). Patterns in the dataset that map the data inputs to the target outputs (correct classifications or levels) may be found, and machine learning model 190 is provided with a mapping that captures these patterns. In some embodiments, machine learning model 190 uses one or more of Gaussian process regression (GPR), Gaussian process classification (GPC), Bayesian neural networks, neural network Gaussian processes, deep belief networks, Gaussian mixture models, or other probabilistic learning methods. Non-probabilistic methods can also be used, including one or more of support vector machines (SVMs), radial basis functions (RBFs), clustering, nearest neighbor algorithms (k-NNs), linear regression, random forests, neural networks (e.g., artificial neural networks), etc. In some embodiments, machine learning model 190 is a multivariate analysis (MVA) regression model.

[0046] The prediction component 114 provides the current characteristic data 146 (e.g., as input) to the trained machine learning model 190 and executes the trained machine learning model 190 (e.g., for the input, to obtain one or more outputs). The prediction component 114 can determine (e.g., extract) predicted data 160 from the trained machine learning model 190 and determine (e.g., extract) uncertainty data that indicates a level of confidence that the predicted data 160 corresponds to the current performance data 156. In some embodiments, the prediction component 114 or the corrective action component 122 uses the uncertainty data (e.g., an uncertainty function, or a retrieval function derived from the uncertainty function) to determine whether to use the predicted data 160 to perform corrective action or whether to further train the model 190.

[0047] For purposes of explanation and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (i.e., prior data, historical characteristic data 144, and historical performance data 154) and providing current characteristic data 146 to one or more trained probabilistic machine learning models 190 to determine predicted data 160. In other implementations, heuristic or rule-based models are used to determine predicted data 160 (e.g., without using a trained machine learning model). In other implementations, non-probabilistic machine learning models may be used. The prediction component 114 monitors the historical characteristic data 144 and the historical performance data 154. In some embodiments, any of the information described with respect to data input 210 of FIG. 2 is monitored or otherwise used in the heuristic or rule-based models.

[0048] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 is provided by fewer machines. For example, in some embodiments, server machines 170 and 180 are combined into a single machine, while in some other embodiments, server machine 170, server machine 180, and prediction server 112 are combined into a single machine. In some embodiments, client device 120 and prediction server 112 are combined into a single machine.

[0049] In general, functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed by prediction server 112 in other embodiments, where appropriate. In addition, functions attributed to particular components may be performed by various or multiple components operating together. For example, in some embodiments, prediction server 112 determines corrective actions based on prediction data 160. In another example, client device 120 determines prediction data 160 based on data received from a trained machine learning model.

[0050] Additionally, the functionality of a particular component may be performed by various or multiple components working together. In some embodiments, one or more of prediction server 112, server machine 170, or server machine 180 are accessed as a service offered to other systems or devices through an appropriate application programming interface (API).

[0051] In some embodiments, a "user" is represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. In some examples, a set of individual users federated as a group of administrators is considered a "user."

[0052] Although embodiments of the present disclosure are discussed with respect to determining predictive data 160 for defect source tracking and defect root cause identification in substrate processing at a production facility (e.g., a substrate processing facility), in some embodiments the present disclosure may also be applied to quality detection generally. Embodiments may be applied generally to determining the quality of parts based on different types of data.

[0053] 2 illustrates a dataset generator 272 (e.g., dataset generator 172 of FIG. 1 ) for creating a dataset for a machine learning model (e.g., model 190 of FIG. 1 ), according to certain embodiments. In some embodiments, dataset generator 272 is part of server machine 170 of FIG. 1 . The dataset generated by dataset generator 272 of FIG. 2 can be used to train a machine learning model (e.g., see FIG. 5B ) and trigger the implementation of corrective actions (e.g., see FIG. 5C ).

[0054] A dataset generator 272 (e.g., dataset generator 172 in FIG. 1 ) creates a dataset for a machine learning model (e.g., model 190 in FIG. 1 ). The dataset generator 272 creates the dataset using historical characteristic data 244 (e.g., historical characteristic data 144 in FIG. 1 ) and historical performance data 254 (e.g., historical performance data 154 in FIG. 1 ). The system 200 in FIG. 2 shows the dataset generator 272, a data input 210, and a target output 220 (e.g., target data).

[0055] In some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs). In some embodiments, the dataset generator 272 does not generate a target output (e.g., for unsupervised learning). In some embodiments, the dataset generator generates one or more target outputs 220 (e.g., for supervised learning) that correspond to the data inputs 210. The dataset may further include mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 are also referred to as “features,” “attributes,” or “information.” In some embodiments, the dataset generator 272 provides the dataset to the training engine 182, the validation engine 184, or the test engine 186, and the dataset is used to train, validate, or test the machine learning model 190.

[0056] In some embodiments, data set generator 272 generates data input 210 and target output 220. In some embodiments, data input 210 includes one or more sets of historical characteristic data 244. In some embodiments, historical characteristic data 244 includes one or more of characteristic data from one or more types of sensors and / or measurement devices, combinations of characteristic data from one or more types of sensors and / or measurement devices, patterns from characteristic data from one or more types of sensors and / or measurement devices, etc.

[0057] In some embodiments, the dataset generator 272 generates a first data input corresponding to a first set of historical feature data 244A for training, validating, or testing a first machine learning model, and the dataset generator 272 generates a second data input corresponding to a second set of historical feature data 244B for training, validating, or testing a second machine learning model.

[0058] In some embodiments, the dataset generator 272 discretizes (e.g., segments) one or more of the data inputs 210 or the target outputs 220 (e.g., for use in a classification algorithm for a regression problem). Discretizing the data inputs 210 or the target outputs 220 (e.g., segmenting by a sliding window) converts continuous values ​​of variables into discrete values. In some embodiments, the discrete values ​​of the data inputs 210 represent discrete historical characteristic data 244 to obtain the target outputs 220 (e.g., discrete historical performance data 254).

[0059] The data inputs 210 and target outputs 220 for training, validating, or testing a machine learning model include information for a particular facility (e.g., for a particular substrate production facility). In some examples, the historical characteristic data 244 and the historical performance data 254 are for the same production facility.

[0060] In some embodiments, the information used to train the machine learning model is from a particular type of production equipment 124 in a production facility having particular characteristics, allowing the trained machine learning model to determine an outcome for a particular group of production equipment 124 based on current parameter inputs (e.g., current characteristic data 146) associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the machine learning model is for components from more than one production facility, allowing the trained machine learning model to determine an outcome for a component based on inputs from one production facility.

[0061] In some embodiments, after generating the dataset and using the dataset to train, validate, or test the machine learning model 190, the machine learning model 190 is further trained, validated, or tested (e.g., current performance data 156 in FIG. 1 ) or adjusted (e.g., adjusting weights associated with the input data of the machine learning model 190, such as connection weights in a neural network).

[0062] 3 is a block diagram illustrating a system 300 for generating predictive data 360 (e.g., predictive data 160 of FIG. 1 ), according to certain embodiments. System 300 is used to determine predictive data 360 via a trained machine learning model (e.g., model 190 of FIG. 1 ) for defect source tracking and defect root cause identification (e.g., for performing corrective actions).

[0063] In block 310, the system 300 (e.g., the prediction system 110 of FIG. 1 ) performs data splitting (e.g., via the dataset generator 172 of the server machine 170 of FIG. 1 ) of historical data (e.g., the historical characteristic data 344 and / or the historical performance data 354 for the model 190 of FIG. 1 ) to generate a training set 302, a validation set 304, and a test set 306. In some examples, the training set is 60% of the historical data, the validation set is 20% of the historical data, and the test set is 20% of the historical data. The system 300 generates multiple sets of features for each of the training set, validation set, and test set. In some examples, if the historical data includes features derived from 20 sensors (e.g., sensor 126 of FIG. 1 , sensors of production equipment and / or measurement equipment) and 100 products (e.g., products each corresponding to characteristic data from the 20 sensors), the first set of features would be sensors 1-10, the second set of features would be sensors 11-20, the training set would be products 1-60, the validation set would be products 61-80, and the test set would be products 81-100. In this example, the first set of features for the training set would be parameters from sensors 1-10 for products 1-60.

[0064] At block 312, system 300 performs model training using training set 302 (e.g., via training engine 182 of FIG. 1 ). In some embodiments, system 300 trains multiple models using multiple sets of features in training set 302 (e.g., a first set of features in training set 302, a second set of features in training set 302, etc.). For example, system 300 trains machine learning models to generate a first trained machine learning model using a first set of features in the training set (e.g., characteristic data from sensors 1-10 of products 1-60) and generate a second trained machine learning model using a second set of features in the training set (e.g., characteristic data from sensors 11-20 of products 1-60). In some embodiments, the first trained machine learning model and the second trained machine learning model are combined to generate a third trained machine learning model (e.g., which, in some embodiments, is a better predictor than either the first or second trained machine learning model alone). In some embodiments, the sets of features used in comparing the models overlap (e.g., a first set of features is characteristic data from sensors 1-15, and a second set of features is characteristic data from sensors 5-20). In some embodiments, hundreds of models are generated, including models with various permutations of features and combinations of models.

[0065] At block 314, the system 300 performs model validation using the validation set 304 (e.g., via the validation engine 184 of FIG. 1 ). The system 300 validates each of the trained models using a corresponding set of features in the validation set 304. For example, the system 300 validates a first trained machine learning model using a first set of features in the validation set (e.g., parameters from sensors 1-10 of products 61-80) and validates a second trained machine learning model using a second set of features in the validation set (e.g., parameters from sensors 11-20 of products 61-80). In some embodiments, the system 300 validates hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 determines the accuracy of each of the one or more trained models (e.g., via model validation) and determines whether one or more of the trained models have an accuracy that meets a threshold accuracy. In response to a determination that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312, where the system 300 performs model training using a different set of features from the training set. In response to a determination that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. The system 300 discards trained machine learning models that have an accuracy that is less than the threshold accuracy (e.g., based on a validation set).

[0066] In block 316, the system 300 performs model selection (e.g., via selection engine 185 of FIG. 1 ) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., selected model 308 based on validation in block 314). In response to a determination that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow returns to block 312, and the system 300 performs model training using a further refined training set corresponding to the further refined set of features to determine the trained model with the highest accuracy.

[0067] At block 318, the system 300 performs model testing (e.g., via test engine 186 of FIG. 1 ) using the test set 306 to test the selected model 308. The system 300 tests the first trained machine learning model using a first set of features in the test set (e.g., characteristic data from sensors 1-10 of products 81-100) and determines that the first trained machine learning model meets a threshold accuracy (e.g., based on the first set of features in the test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 overfits the training set 302 and / or the validation set 304 and cannot be applied to other datasets, such as the test set 306), flow proceeds to block 312, where the system 300 performs model training (e.g., retraining) using a different training set corresponding to a different set of features (e.g., characteristic data from a different sensor). In response to a determination that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model learns patterns in past data to make predictions, and in block 318, the system 300 applies the model to the remaining data (e.g., the test set 306) to test the predictions.

[0068] In block 320, the system 300 uses the trained model (e.g., selected model 308) to receive current characteristic data 346 (e.g., current characteristic data 146 of FIG. 1 ) and determine (e.g., extract) from the trained model predicted data 360 (e.g., predicted data 160 of FIG. 1 ) for defect source tracking and defect root cause identification and to perform corrective actions. In some embodiments, the current characteristic data 346 corresponds to the same feature types in the historical characteristic data 344. In some embodiments, the current characteristic data 346 corresponds to the same feature types as a subset of the feature types in the historical characteristic data 344 used to train the selected model 308.

[0069] In some embodiments, current data is received. In some embodiments, the current data includes current performance data 356 (e.g., current performance data 156 of FIG. 1 ) and / or current characteristic data 346. In some embodiments, at least a portion of the current data is received from a measurement instrument (e.g., measurement instrument 128 of FIG. 1 ) or via user input. In some embodiments, model 308 is retrained based on the current data. In some embodiments, a new model is trained based on current performance data 356 and current characteristic data 346.

[0070] In some embodiments, one or more of blocks 310-320 are performed in various orders and / or with other operations not presented and described herein. In some embodiments, one or more of blocks 310-320 are not performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, and / or model testing of block 318 are not performed.

[0071] 4A is a block diagram illustrating defect categories 410A-410D, according to certain embodiments. The characteristic data may include different types of characteristic data 400A-400Z (e.g., different subsets of characteristic data) from one or more types of sensors and / or one or more types of metrology instruments. In some embodiments, as shown in FIG. 4A, a processing device may identify regions of a substrate that correspond to defect categories 410A-410D based on characteristic data 400A (e.g., a first subset of characteristic data 400A).

[0072] In some embodiments, the characteristic data 400A may include SEM images captured by an SEM metrology instrument, EDX images captured by an EDX metrology instrument, etc., morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer data, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc. The characteristic data 400A may be associated with (e.g., captured from) various regions of the substrate. Based on the characteristic data 400A, a processing device may determine that a particular region of the substrate contains defects of a particular defect category 410A-410D.

[0073] The defect categories 410 may include one or more of line breaks, non-fills, bridges, stains, scratches, glass damage, foreign matter (particles), residues, resist collapse, z-axis defects, large areas, via stress, voids, pits, crystalline defects, cracks, etc.

[0074] 4B is a block diagram illustrating defect subcategories, according to certain embodiments. In some embodiments, as shown in FIG. 4B, the processing device subdivides regions of the substrate corresponding to defect category 410A into defect subcategories 410A1-410A4, regions of the substrate corresponding to defect category 410B into defect subcategories 410B1-410B3, and regions of the substrate corresponding to defect category 410C into defect subcategories 410C1-410C4 based on characteristic data 400B-400D (e.g., a second subset of characteristic data).

[0075] Defect subcategories include spherical particles, random particles, rod-shaped particles, flake particles, post-imprint fall-on particles, ring pits, micropits, macropits, mouse bites, multi-line bridging, CMP bridging, photoresist bridging, microbridging, organic staining, inorganic staining, lithography pitting, step bunching, stacking faults, shallow triangles, obtuse triangles, surface triangles, downfalls, tics, chatter marks, crescents, micropipes, PL circles, basal plane dislocations, protrusions, hillocks, and more.

[0076] In some embodiments, characteristic data 400B-400C can include one or more of an SEM image, an EDX image, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layers, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc. In some embodiments, characteristic data 400A includes an SEM image, characteristic data 400B includes a defect map, characteristic data 400C includes EDX data (e.g., EDX images generated via X-ray fluorescence (XRF), inductively coupled plasma mass spectrometry (ICP-MS), triple quadrupole ICP-MS (ICP-QQQ), etc.), and characteristic data 400D includes a chip layout.

[0077] 4C is a block diagram associated with defect subcategories, according to certain embodiments. In some embodiments, as shown in FIG. 4C , the processing device identifies a subset of defect subcategories (e.g., defect subcategories of interest) that correspond to performing corrective actions. In some embodiments, the subset of defect subcategories is determined based on user input 444 indicating one or more subcategories of interest. In some embodiments, the defect subcategories are determined based on machine learning. In some embodiments, the defect subcategories are determined based on a threshold.

[0078] In some embodiments, the processing device can compile defect data 420 and location data 430 to track each defect by layer or operation in the substrate production process. For example, in some embodiments, the processing device can identify the defect (e.g., characteristic data) and, based on the defect evolution information, identify the previous process operation where the defect source or defect root cause occurred. The location data can include wafer mapping coordinates and identification (e.g., layer number or process operation number) to indicate the location and layer / operation of such data on the substrate. For example, the location data can be stored using wafer mapping (e.g., KLA Results File (KLARF)). In some embodiments, a user identifies a subset of defect subcategories (e.g., 410A1, 410A2, 410A4, 410B1, 410B2, 410C1, 410C2, 410C4) via user input 444. Furthermore, the processing device can add location data corresponding to each identified defect subcategory to the location data 430 of the next or subsequent layer. Processing logic identifies regions corresponding to the identified defect subcategories and identifies further characteristic data for the regions, including location data.

[0079] In some embodiments, one of the defect subcategories may be selected, and defect mapping coordinates of the corresponding region of the defect subcategory may be added to corresponding position data (e.g., defect map) of one or more subsequent layers to generate position data (e.g., defect map) for the subsequent layer (e.g., KLA Results File (KLARF), etc.). In some embodiments, a portion of the defect mapping coordinates of the corresponding region of the defect subcategory may be selected, and the selected portion of the defect mapping coordinates of the corresponding region of the defect subcategory may be added to the defect map of the subsequent layer.

[0080] In some embodiments, location data for a subsequent layer of the substrate can be identified. The location data can correspond to defect mapping coordinates for a previously selected defect subcategory in a corresponding region of the defect subcategory. In some embodiments, a defect evolution database entry can be created that includes characteristic data from each layer, including one or more defect images, the location data, and defect evolution information for the defects, including one or more defect evolution images. In some embodiments, the defect evolution database entry can be stored for subsequent access. In some embodiments, characteristic data for a layer of the substrate can be identified, and based on the defect evolution database entry, the characteristic data can be matched with a database entry for defect evolution information. Based on the defect evolution information, the defect can be traced to a defect source or a defect root cause can be identified, and based on the defect evolution information, the execution of a corrective action associated with the substrate processing system can be triggered. In some embodiments, the defect source or defect root cause may have arisen due to a defect or deficiency in a previous process corresponding to a previous layer of the substrate production process.

[0081] A defect that causes a substrate to fail is sometimes referred to as killing the substrate. A kill ratio may be the rate at which substrates with a certain type of defect suffer failure. A defect that does not affect the functionality of the substrate has a low kill ratio. For example, a low kill ratio may result if a defect from a first substrate processing operation is etched away from the substrate in a second substrate processing operation. A defect that often interferes with the proper function of the substrate has a high kill ratio. For example, a high kill ratio may result if a defect from a substrate processing operation is not removed by a subsequent operation. A defect with a low kill ratio may be more tolerable than a defect with a high kill ratio. In some embodiments, a defect subcategory may include defects that do not cause the performance data 152 of the produced substrate to fall below a certain threshold (e.g., a low kill ratio, etc.).

[0082] FIG. 4D is a block diagram illustrating substrate defects (e.g., adder defects and common defects) according to certain embodiments. In some embodiments, as shown in FIG. 4D, a processing device identifies characteristic data 400A (e.g., a first subset of characteristic data 400) for substrate 401. In some embodiments, substrate 401 is undergoing operation 1 of a production process. In some embodiments, substrate 402 may be substrate 401 after undergoing operation 2 of a production process. In some embodiments, substrate 403 may be substrate 402 after undergoing operation n of a production process. In some embodiments, characteristic data 400A includes location data 400B1 for regions of the substrate corresponding to defect categories 450A-450C. Such location data 400B1 can be identified as adder defects (e.g., new defects or defects not carried over from a previous layer or operation in the substrate production process). In some embodiments, as shown in FIG. 4D, a processing device can identify regions of the substrate corresponding to defect categories 450A-450C based on characteristic data 400A (e.g., a first subset of characteristic data 400). In some embodiments, processing logic maps areas of the substrate corresponding to defect categories as position data 400B1 (eg, defect mapping, KLARF, etc.).

[0083] In some embodiments, a subsequent operation (e.g., operation 2) is performed on the substrate 401. In some embodiments, the processing device identifies characteristic data 400E (e.g., a second subset of characteristic data 400) for the substrate 402 (e.g., the substrate 401 after undergoing operation 2 of the substrate production process). In some embodiments, the characteristic data 400E includes location data 400B2 for regions of the substrate corresponding to defect categories 460A-460C. In some embodiments, as shown in FIG. 4D , the processing device can identify regions of the substrate corresponding to defect categories 460A-460C based on the characteristic data 400E (e.g., the second subset of characteristic data 400). In some embodiments, the location data 400B2 can identify adder defects (e.g., new defects or defects that are not carried over from a previous layer or operation in the substrate production process). In some embodiments, the location data 400B1 can identify common defects (e.g., old defects or defects that are carried over from a previous layer or operation in the substrate production process). In some embodiments, processing logic does not map regions of the substrate corresponding to defect categories that are common defects. In some embodiments, the processing device maps regions of the substrate corresponding to defect categories as position data 400B2 (e.g., defect mapping, KLARF, etc.).

[0084] In some embodiments, a subsequent operation (e.g., operation n) is performed on the substrate 402. In some embodiments, the processing device identifies characteristic data 400F (e.g., a third subset of characteristic data 400) for the substrate 403 (e.g., the substrate 402 after undergoing operation n of the substrate production process). In some embodiments, the characteristic data 400F includes location data 400B3 for regions of the substrate corresponding to defect categories 470A-470C. In some embodiments, as shown in FIG. 4D , the processing device can identify regions of the substrate corresponding to defect categories 470A-470C based on the characteristic data 400F (e.g., the third subset of characteristic data 400). In some embodiments, the location data 400B3 can identify adder defects (e.g., new defects or defects that are not carried over from a previous layer or operation in the substrate production process). In some embodiments, the location data 400B1-400B2 can identify common defects (e.g., old defects or defects that are carried over from a previous layer or operation in the substrate production process). In some embodiments, processing logic does not map regions of the substrate corresponding to defect categories that are common defects. In some embodiments, processing logic maps regions of the substrate corresponding to defect categories as position data 400B3 (e.g., defect mapping, KLARF, etc.).

[0085] 4C , a subset of defect subcategories (e.g., 450A of operation 1) may be identified (e.g., via user input 444, via a processing device). Further, the processing device may add location data corresponding to each identified defect subcategory (e.g., 450A) corresponding to a region of the substrate to the location data 430 of the next layer (e.g., operation 2), even if such location data may be common defects. In some embodiments, by adding location data corresponding to identified defect subcategories corresponding to a region of the substrate to the location data of the next layer, defect evolution may be observed operation by operation and layer by layer.

[0086] In some embodiments, characteristic data corresponding to the selected defect subcategory (e.g., defect images, mapping coordinates of corresponding regions of the defect subcategory, etc.) may be identified (e.g., via user input 444, via the processing device). In some embodiments, the processing device may create database entries (defect database entries, defect evolution database entries, etc.) that include the characteristic data from each layer (e.g., one or more defect images and defect evolution information for the defects including one or more defect evolution images). In some embodiments, the defect evolution images may include multiple images of defects from multiple layers of the substrate that correspond to separate operations in the production process. The processing device may store the database entries in a database (e.g., a defect database, a defect evolution database, a knowledge database, etc.) for subsequent access.

[0087] In some embodiments, the processing device can identify characteristic data for a substrate processed by the substrate processing system. In some embodiments, the processing device can identify regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data. In some embodiments, the processing device can subdivide the regions of the substrate corresponding to the first defect category into defect subcategories based on a second subset of the characteristic data. In some embodiments, the processing device can trigger the execution of a corrective action associated with the substrate processing system based on one or more of the regions corresponding to at least one of the defect subcategories. In some embodiments, the processing device can determine a defect source (e.g., defect source tracking) based on at least one of the defect subcategories, the corrective action corresponding to the defect source. In some embodiments, the processing device can determine a defect root cause (e.g., defect root cause identification) based on at least one of the defect subcategories, the corrective action corresponding to the defect root cause. In some embodiments, the defect source or defect root cause can correspond to a previous layer of the substrate and / or a previous process operation in the substrate production process. In some embodiments, the processing device can determine the defect source and / or defect root cause based on the defect evolution information. In some embodiments, the defect evolution information may be stored in a defect evolution database.

[0088] In some embodiments, the processing device may search a database (e.g., a defect evolution database) for a database entry with past characteristic data that is substantially similar (e.g., most similar) to the current characteristic data (e.g., the defect image) and match the current characteristic data corresponding to the defect with the substantially similar defect evolution entry (e.g., the most similar defect evolution database entry). In some embodiments, the processing device may correlate the search results (e.g., the substantially similar defect evolution entry, the most similar database entry) with, for example, the defect source, the defect root cause, and / or the corrective action within a particular confidence metric. In some embodiments, the processing device may identify the defect source, the defect root cause, and / or the corrective action. In some embodiments, the defect source and / or the defect root cause may correspond to a prior operation in the substrate production process. In some embodiments, the defect source, the defect root cause, and / or the corrective action may correspond to production equipment previously used in the substrate production process.

[0089] In some embodiments, the prior operations in the substrate production process may be wet cleaning, surface passivation, photolithography, ion implantation, etching, dry etching, reactive ion etching (RIE), deep reactive ion etching, atomic layer etching (ALE), wet etching, buffered oxide etching, plasma ashing, thermal treatment, rapid thermal annealing, furnace annealing, thermal oxidation, chemical vapor deposition (CVD), atomic layer deposition (ALD), physical vapor deposition (PVD), molecular beam epitaxy (MBE), laser lift-off, electrochemical deposition (ECD), chemical mechanical polishing (CMP), wafer testing, die preparation, through-silicon via production, wafer mounting, wafer backgrinding and polishing, wafer bonding and lamination, redistribution layer production, wafer bumping, die cutting or wafer dicing, IC packaging, die attach, molding, baking, electroplating, laser marking or silkscreen printing, IC testing, etc.

[0090] In some embodiments, during a defect out-of-control (OOC) event in a substrate production system (e.g., a substrate production facility), a processing device can identify characteristic data corresponding to the OOC defects. The processing device can search a database (e.g., a knowledge database) for a database entry that is most similar to the characteristic data (e.g., a defect image or a defect evolution image) and match the characteristic data corresponding to the OOC defect with the most similar database entry. The processing device can correlate the search results (e.g., the most similar database entry) with, for example, the defect source, the defect root cause, and / or a corrective action within a particular confidence metric. In some embodiments, the processing device or a user can review the recommendations and develop an action plan.

[0091] 5A-5C are flow diagrams of methods 500A-500C related to defect analysis, according to certain embodiments. In some embodiments, methods 500A-500C are performed by processing logic including hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 500A-500C are performed, at least in part, by prediction system 110. In some embodiments, method 500A is performed by client device 120 (e.g., corrective action component 122) and / or prediction system 110 (e.g., prediction component). In some embodiments, method 500B is performed by server machine 180 (e.g., training engine 182, etc.). In some embodiments, method 500C is performed by prediction server 112 (e.g., prediction component 114) and / or client device 120 (e.g., corrective action component 122). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, client device 120, etc.), cause the processing device to perform one or more of methods 500A-500C.

[0092] For ease of explanation, methods 500A-500C are illustrated and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or simultaneously, with other operations not shown and described herein. Furthermore, in some embodiments, not all illustrated operations are performed to implement methods 500A-500C in accordance with the subject matter of the present disclosure. Additionally, those skilled in the art will understand and appreciate that methods 500A-500C may alternatively be represented as a series of interrelated states via a state diagram or events.

[0093] FIG. 5A is a flow diagram of a method 500A for defect analysis (eg, defect source tracking and defect root cause identification) in accordance with certain embodiments.

[0094] 5A, in some embodiments, at block 502, processing logic performing method 500A identifies characteristic data (e.g., characteristic data 142 of FIG. 1) of a substrate processed by a substrate processing system (e.g., production equipment 124 of FIG. 1). The substrate may be a wafer, a semiconductor, a display, etc.

[0095] In some embodiments, the characteristic data may be at least one of an SEM image, an EDX image, etc. In some embodiments, the characteristic data may be at least one of morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc.

[0096] The characteristic data may be captured via a metrology tool after one or more substrate processing operations.

[0097] The characteristic data may be captured via the sensors after one or more substrate processing operations.

[0098] At block 504, processing logic identifies regions of the substrate corresponding to a first defect category based on the first subset of characteristic data. In some embodiments, the first subset of characteristic data 142 may be at least one of an SEM image, an EDX image, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, defect metadata including, but not limited to, gray level data and signal-to-noise data, etc. In some embodiments, the first subset of characteristic data 142 may be at least one of an SEM image, an EDX image, etc. The defect categories may include one or more of a line break, a non-fill, a bridge, a stain, a scratch, glass damage, a foreign object (particle), a residue, a resist collapse, a via stress, a void, a pit, a crystalline defect, a crack, etc.

[0099] At block 506, the processing logic subdivides regions of the substrate corresponding to the first defect category into defect subcategories based on the second subset of characteristic data. The defect subcategories may include one or more of spherical particles, random particles, rod-shaped particles, flake particles, post-imprint fall-on particles, ring pits, micropits, macropits, mouse bites, multi-line bridging, CMP bridging, photoresist bridging, microbridging, organic staining, inorganic staining, lithography pitting, step bunching, stacking faults, shallow triangles, obtuse triangles, surface triangles, downfalls, tics, chatter marks, crescents, micropipes, PL circles, basal plane dislocations, protrusions, hillocks, etc.

[0100] In some embodiments, processing logic may use a machine learning model to perform one or more of blocks 502-508 (see, for example, Figures 5B-5C).

[0101] At block 508, processing logic causes execution of a corrective action associated with the substrate processing system based on at least one of the defect subcategories. In some embodiments, block 508 includes determining a defect root cause based on at least one of the plurality of defect subcategories, the corrective action corresponding to the defect root cause. For example, the corrective action corresponding to the defect root cause can be a corrective action that corrects an aspect of a particular subset of production equipment that caused the defect. Tracing the root cause can be determining that the defect is caused by a particular subset of production equipment based on substrate characteristic data. In some embodiments, the defect root cause can be a particular piece of substrate processing equipment (e.g., production equipment 124 of FIG. 1 ).

[0102] In some embodiments, block 508 includes determining a defect source based on at least one of a plurality of defect subcategories (e.g., a corrective action corresponds to the defect source). For example, a corrective action is consistent with a defect source if execution of the corrective action prevents further defects from occurring at the defect source. In one example, the corrective action may be replacing a component of production equipment that caused the defect. In some embodiments, the defect source may be identified using defect source tracking. Defect source tracking may be determining the source of the defect based on the defect characteristic data. In some embodiments, the defect source may be a component of substrate processing equipment (e.g., production equipment 124 of FIG. 1 ) that should be cleaned, repaired, and / or replaced.

[0103] In some embodiments, the corrective action includes issuing an alert, triggering a cleaning action, triggering a repair action, triggering a replacement of a component, shutting down one or more portions of the substrate processing equipment, determining a predicted end of life for a component of the substrate processing equipment, etc. In some embodiments, the corrective action component 122 of FIG. 1 receives an instruction for a corrective action (e.g., based on the predictive data 160) from the predictive system 110 and triggers the execution of the corrective action.

[0104] In some embodiments, the processing logic further determines that at least one of the defect subcategories corresponds to a defect evolution associated with the execution of a corrective action. In some embodiments, the defect evolution may be information about defects in multiple operations (e.g., layers) within a production process of the substrate. The defect evolution may track defects by layer and document changes to the defects (e.g., characteristic data). The defect conversion uses the defect evolution information to track defects by layer. In some embodiments, the determining is based on the defect evolution information of at least one of the defect subcategories. In some embodiments, the defect evolution information may be characteristic data 142 (e.g., historical characteristic data 144, current characteristic data 146) of a substrate that indicates the same coordinates in various layers of the substrate (e.g., at least one of the layers has a defect or anomaly at that coordinate). In some embodiments, a kill ratio associated with at least one of the plurality of defect subcategories may be determined using the defect evolution data. In some embodiments, the kill ratio may be a percentage of defects estimated to cause substrate failure (e.g., die failure on the substrate). For example, if defects are found in a certain number of dies, a certain number of those dies will fail. The ratio of the number of dies with defects to the number of failed dies may be a kill ratio for such defects. In some embodiments, corrective action associated with the substrate processing system may or may not be taken based on the defect evolution information and / or the associated kill ratio.

[0105] In some embodiments, the processing logic further identifies a first subset of defect subcategories (e.g., including at least one of the defect subcategories corresponding to a corrective action). The first subset of defect subcategories corresponds to substrate characteristic data that meets a threshold level (e.g., a thickness that meets a threshold thickness, a conductivity that meets a threshold conductivity, etc.). In some embodiments, the threshold level can correspond to characteristic data (e.g., an SEM image, an EDX image, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, or defect metadata including, but not limited to, gray level data and signal-to-noise data, etc.) or performance data (e.g., one or more values ​​of image data, morphology data, size attribute data, dimensional attribute data, temperature, spacing, current, power, voltage, etc.).

[0106] In some embodiments, identifying the first subset of defect subcategories is based on one or more of: user input indicating one or more subcategories of interest (e.g., user input 444 in FIG. 4C ), or predictive data associated with output received from a trained machine learning model based on data input including historical defect subcategories and / or target output including historical characteristic data.

[0107] In some embodiments, the processing logic further provides the first subset of the characteristic data as an input to a trained machine learning model. The processing logic obtains an output from the trained machine learning model related to predicted data. Identifying regions of the substrate corresponding to the first defect category is based on the predicted data.

[0108] In some embodiments, the processing logic further provides the second subset of the characteristic data as an input to the trained machine learning model. The processing logic further obtains output from the trained machine learning model related to predicted data. The subcategorization of the plurality of regions into defect subcategories is based on the predicted data.

[0109] In some embodiments, the defect source and / or defect root cause may be formed during a prior operation in the substrate production process, hi some embodiments, the prior operation in the substrate production process may be correlated to a prior layer or the same layer in the substrate production process. In some embodiments, the preceding operation may be one of wet cleaning, surface passivation, photolithography, ion implantation, etching, dry etching, reactive ion etching (RIE), deep reactive ion etching, atomic layer etching (ALE), wet etching, buffered oxide etching, plasma ashing, thermal treatment, rapid thermal annealing, furnace annealing, thermal oxidation, chemical vapor deposition (CVD), atomic layer deposition (ALD), physical vapor deposition (PVD), molecular beam epitaxy (MBE), laser lift-off, electrochemical deposition (ECD), chemical mechanical polishing (CMP), wafer testing, die preparation, through-silicon via production, wafer mounting, wafer backgrinding and polishing, wafer bonding and lamination, redistribution layer production, wafer bumping, die cutting or wafer dicing, IC packaging, die attach, molding, baking, electroplating, laser marking or silkscreen printing, IC testing, and the like.

[0110] In some embodiments, the defect source or defect root cause may originate from a previous layer in the substrate production process. In some embodiments, the previous layer may correspond to a previous operation in the substrate production process. For example, in some embodiments, the defect may occur in operation 1 of the substrate 401. The characteristic data (e.g., characteristic data 142, past characteristic data 144) may be identified after operation n of the substrate 403. The defect source or defect root cause may be identified based on the characteristic data 400F of the substrate 403 and the defect progression entries in the defect evolution database. In further embodiments, the processing device may trigger corrective action based on the defect source or defect root cause identified based on the defect progression entries in the defect evolution database.

[0111] FIG. 5B is a method for training a machine learning model (e.g., model 190 of FIG. 1) to determine predictive data (e.g., predictive data 160 of FIG. 1) for defect source tracking and defect root cause identification.

[0112] 5B , at block 510 of method 500B, processing logic identifies past characteristic data for the substrate (e.g., past characteristic data 144 of FIG. 1 , past input characteristic data). The past characteristic data can include data from past substrates such as image data, SEM images, EDX images, morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, or defect metadata including, but not limited to, gray level data and signal-to-noise data, temperature, spacing, current, power, voltage, etc.

[0113] In some embodiments, at block 512, processing logic identifies historical performance data for the substrate (e.g., historical performance data 154 of FIG. 1 ). The historical performance data may include data from past substrates, such as image data, morphology data, size attribute data, dimensional attribute data, values ​​for one or more of temperature, spacing, current, power, voltage, etc. Performance data, including historical performance data, may include metrology data or user input indicative of the performance of the substrate in meeting particular parameters or achieving a particular level of performance (e.g., the ability to pass a probe test that measures voltage). At least a portion of the historical characteristic data and historical performance data may be associated with new substrate processing equipment parts (e.g., used for benchmark testing). At least a portion of the historical characteristic data and historical performance data may be associated with produced substrates.

[0114] At block 514, processing logic trains a machine learning model using data inputs including historical characteristic data 144 and / or target outputs including historical performance data 154 to generate a trained machine learning model.

[0115] In some embodiments, the historical property data is of a past substrate and / or the historical performance data corresponds to a past substrate. In some embodiments, the historical property data includes a past image of a past substrate and / or the historical performance data corresponds to a past substrate. The historical performance data can be related to substrate quality, such as substrate metrology data, substrate throughput, substrate defects, etc. The historical performance data can be related to substrate processing equipment component quality, such as test data, substrate metrology data, substrate time to failure, etc.

[0116] At block 514, the machine learning model may be trained using data inputs including historical characteristic data and / or target outputs including historical performance data to generate a trained machine learning model configured to identify a source or root cause of the identified defect based on the characteristic data (e.g., the characteristic data of block 502 of FIG. 5A ). At block 508 of FIG. 5A , processing logic uses the source or root cause identified via the trained machine learning model to trigger the execution of a corrective action associated with the substrate processing system.

[0117] In some embodiments, the historical property data in block 510 is for a past substrate, and the historical performance data in block 512 corresponds to a past substrate.

[0118] At block 514, the machine learning model may be trained using data inputs including historical characteristic data 144 and target outputs including historical characteristic data 144 to generate a trained machine learning model configured to predict performance data 152 (e.g., performance data of a substrate, predicted data 160) based on the characteristic data 144 (e.g., characteristic data of block 502 of FIG. 5A). In response to the predicted performance data satisfying a first threshold (e.g., a threshold kill ratio of defects of interest (DOIs)), processing logic may trigger a corrective action (e.g., shutdown, cleaning, repair, replacement, etc. of a substrate processing equipment component). In response to the predicted performance data satisfying a second threshold (e.g., a kill ratio of DOIs), processing logic may cause the substrate processing equipment component to be put into use in the substrate processing system.

[0119] In some embodiments, the historical characteristic data 144 of block 510 includes historical characteristic data 144 of past substrates, and the historical performance data 144 of block 512 includes historical characteristic data 144 .

[0120] At block 514, the machine learning model may be trained using data inputs including the historical characteristic data 144 and target outputs including the historical characteristic data 144 to generate a trained machine learning model configured to predict performance data 152 (e.g., performance data of a substrate processing equipment component) based on the characteristic data 144 (e.g., the characteristic data of blocks 502 and 504 of FIG. 5A). In response to the predicted performance data satisfying a first threshold, the processing logic may trigger a corrective action (e.g., shutting down, cleaning, repairing, or replacing the substrate processing equipment component). In response to the predicted performance data satisfying a second threshold, the processing logic may cause the substrate processing equipment component to be used in the substrate processing system.

[0121] FIG. 5C is a method 500C for triggering the execution of corrective actions using a trained machine learning model for defect analysis (e.g., model 190 of FIG. 1).

[0122] 5C, at block 520 of method 500C, processing logic identifies characteristic data 144. In some embodiments, characteristic data 144 at block 510 includes an image of the substrate.

[0123] At block 522, processing logic provides the feature data 144 as data input to a trained machine learning model (e.g., trained via block 514 of FIG. 5B).

[0124] At block 524, processing logic receives output from the trained machine learning model related to the predicted data.

[0125] At block 526, processing logic causes the execution of corrective actions based on the predictive data 160.

[0126] In some embodiments, the characteristic data 144 is images of the substrate, and the trained machine learning model of block 522 was trained using data inputs including past images of the substrate and target outputs including past performance data 154 (e.g., substrate quality using past substrate processing equipment parts).

[0127] In some embodiments, the characteristic data 144 is an image of the substrate, and the trained machine learning model of block 522 was trained using a data input including historical images of the substrate and a target output including historical performance data 154 including historical characteristic data 144 of defects corresponding to the historical substrate. The predicted data 160 of block 524 may be associated with the predicted performance data (e.g., performance data of the substrate) based on the image. In response to the predicted performance data meeting a first threshold (e.g., kill ratio), the processing logic may trigger a corrective action (e.g., shutdown, cleaning, repair, or replacement of a substrate processing equipment component). In response to the predicted performance data meeting a second threshold (e.g., kill rate), the processing logic may cause the substrate processing equipment component to be used in the substrate processing system.

[0128] In some embodiments, block 502 of FIG. 5A includes training a machine learning model to identify characteristic data of substrates processed by the substrate processing system and using the trained machine learning model to identify characteristic data of substrates processed by the substrate processing system.

[0129] In some embodiments, block 504 of FIG. 5A includes training a machine learning model to identify a plurality of regions of the substrate corresponding to a first defect category based on the first subset of the characteristic data, and using the trained machine learning model to identify a plurality of regions of the substrate corresponding to the first defect category based on the first subset of the characteristic data.

[0130] In some embodiments, block 506 of FIG. 5A includes training a machine learning model to subdivide the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on the second subset of the characteristic data, and using the trained machine learning model to subdivide the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on the second subset of the characteristic data.

[0131] In some embodiments, block 508 of FIG. 5A includes training a machine learning model to trigger the performance of a corrective action associated with the substrate processing system based on at least one of the plurality of defect subcategories, and using the trained machine learning model to trigger the performance of a corrective action associated with the substrate processing system based on at least one of the plurality of defect subcategories.

[0132] In some embodiments, processing logic trains a machine learning model to identify subcategories of objects and uses the trained machine learning model to identify subcategories of objects.

[0133] 6 is a block diagram illustrating a computer system 600, according to certain embodiments. In some embodiments, computer system 600 is one or more of client device 120, prediction system 110, server machine 170, server machine 180, prediction server 112, etc.

[0134] In some embodiments, computer system 600 is connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). In some embodiments, computer system 600 operates as a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. In some embodiments, computer system 600 is provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Furthermore, the term "computer" is intended to include any collection of computers that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methods described herein.

[0135] In a further aspect, computer system 600 includes a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 618, which communicate with each other via a bus 608.

[0136] In some embodiments, the processing device 602 is provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIM) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of instruction set types), or a special-purpose processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0137] In some embodiments, computer system 600 further includes a network interface device 622 (e.g., coupled to a network 674). In some embodiments, computer system 600 further includes a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generating device 620.

[0138] In some implementations, the data storage device 618 includes a non-transitory computer-readable storage medium 624 that stores instructions 626 encoding any one or more of the methods or functions described herein, including instructions encoding the components of FIG. 1 (e.g., the corrective action component 122, the prediction component 114, etc.) and instructions for performing the methods described herein (e.g., one or more of methods 500A-500C).

[0139] In some embodiments, the instructions 626 also reside, completely or partially, within the volatile memory 604 and / or within the processing device 602 during execution by the computer system 600; and thus, in some embodiments, the volatile memory 604 and the processing device 602 also constitute machine-readable storage media.

[0140] Although computer-readable storage medium 624 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that can store or encode a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0141] In some embodiments, the methods, components, and features described herein are implemented by discrete hardware components or are integrated into the functionality of other hardware components, such as an ASIC, FPGA, DSP, or similar device. In some embodiments, the methods, components, and features are implemented by firmware modules or functional circuitry within a hardware device. In some embodiments, the methods, components, and features are implemented in any combination of hardware devices and computer program components, or in a computer program.

[0142] Unless otherwise specified, terms such as "identify," "subcategorize," "cause," "provide," "obtain," "determine," "mask," "resize," "execute," "convert," "apply," "associate," "compare," "train," "receive," "update," and the like refer to actions and processes performed or implemented by a computer system that manipulate data represented as physical (electronic) quantities in computer system registers and memory and convert it into other data similarly represented as physical quantities in computer system memory or registers, or other such information storage, transmission, or display devices. In some embodiments, the terms "first," "second," "third," "fourth," and the like, as used herein, are meant as labels to distinguish between different elements and do not have any ordering meaning due to their numerical symbolic designation.

[0143] The examples described herein also relate to apparatus for performing the methods described herein. In some embodiments, the apparatus comprises a general-purpose computer system that is specially constructed to perform the methods described herein or that is selectively programmed by a computer program stored on the computer system. Such a computer program is stored on a computer-readable tangible storage medium.

[0144] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. In some embodiments, various general-purpose systems are used in accordance with the teachings described herein. In some embodiments, more specialized apparatus are constructed to perform the methods described herein and / or each of their individual functions, routines, subroutines, or operations. Examples of structures for various of these systems are set forth in the description above.

[0145] The above description is illustrative, and not limiting. While the present disclosure has been described with reference to particular illustrative examples and embodiments, it will be recognized that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. Identifying characteristic data of a substrate processed by the substrate processing system; identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data; subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data; triggering the performance of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories; A method comprising:

2. The method of claim 1 , wherein the first subset of the characteristic data comprises at least one of a scanning electron microscope (SEM) image or an energy dispersive X-ray microanalysis (EDX) image.

3. 10. The method of claim 1, wherein the second subset of characteristic data includes at least one of morphology data, size attribute data, dimensional attribute data, defect distribution data, spatial location data, elemental analysis data, wafer signature data, chip layer, chip layout data, edge data, or defect metadata.

4. providing the first subset of the characteristic data as an input to a trained machine learning model; obtaining, from the trained machine learning model, an output related to predicted data, wherein the identifying the plurality of regions of the substrate corresponding to the first defect category is based on the predicted data; and The method of claim 1 further comprising:

5. providing the second subset of the characteristic data as an input to a trained machine learning model; obtaining an output from the trained machine learning model related to predicted data, wherein the subcategorization of the plurality of regions into the plurality of defect subcategories is based on the predicted data; and The method of claim 1 further comprising:

6. causing the implementation of the corrective action; determining a defect source based on the at least one of the plurality of defect subcategories, wherein the corrective action corresponds to the defect source; or determining a defect root cause based on the at least one of the plurality of defect subcategories, wherein the corrective action corresponds to the defect root cause. The method of claim 1 , comprising one or more of:

7. The method of claim 6 , wherein at least one of the defect source or the defect root cause is formed during a prior operation of a substrate production process.

8. The method of claim 6 , wherein at least one of the defect sources or defect root causes arises from a previous layer deposited by a substrate manufacturing process.

9. 10. The method of claim 1, further comprising: identifying a first subset of the plurality of defect subcategories that includes the at least one of the plurality of defect subcategories, the first subset of the plurality of defect subcategories corresponding to substrate characteristic data that meets a threshold level.

10. identifying the first subset of the plurality of defect subcategories User input indicating one or more subcategories of interest; or Prediction data relating to the output received from the trained machine learning model based on the data input including the historical defect subcategories and the target output including the historical characteristic data. The method of claim 9, based on one or more of:

11. determining that the at least one of the plurality of defect subcategories corresponds to a defect evolution associated with the performance of the corrective action; The method of claim 1 further comprising:

12. A non-transitory computer-readable storage medium storing instructions that, when executed, cause a processing device to: Identifying characteristic data of a substrate processed by the substrate processing system; identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data; subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data; triggering the performance of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories; A non-transitory computer-readable storage medium for causing a computer to perform operations including:

13. The operation is providing the first subset of the characteristic data as an input to a trained machine learning model; obtaining, from the trained machine learning model, an output related to predicted data, wherein the identifying the plurality of regions of the substrate corresponding to the first defect category is based on the predicted data; and 13. The non-transitory computer-readable storage medium of claim 12, further comprising:

14. The operation is providing the second subset of the characteristic data as an input to a trained machine learning model; obtaining an output from the trained machine learning model related to predicted data, wherein the subcategorization of the plurality of regions into the plurality of defect subcategories is based on the predicted data; and 13. The non-transitory computer-readable storage medium of claim 12, further comprising:

15. causing the implementation of the corrective action; determining a defect source based on the at least one of the plurality of defect subcategories, wherein the corrective action corresponds to the defect source; or determining a defect root cause based on the at least one of the plurality of defect subcategories, wherein the corrective action corresponds to the defect root cause.

13. The non-transitory computer-readable storage medium of claim 12, comprising one or more of:

16. The operation is identifying a first subset of the plurality of defect subcategories including the at least one of the plurality of defect subcategories, the first subset of the plurality of defect subcategories corresponding to substrate characteristic data that meets a threshold level.

13. The non-transitory computer-readable storage medium of claim 12, further comprising:

17. identifying the first subset of the plurality of defect subcategories User input indicating one or more subcategories of interest; or Prediction data relating to the output received from the trained machine learning model based on the data input including the historical defect subcategories and the target output including the historical characteristic data.

17. The non-transitory computer-readable storage medium of claim 16, based on one or more of:

18. Memory and a processing device coupled to the memory, the processing device comprising: Identifying characteristic data of a substrate processed by the substrate processing system; identifying a plurality of regions of the substrate corresponding to a first defect category based on a first subset of the characteristic data; subcategorizing the plurality of regions of the substrate corresponding to the first defect category into a plurality of defect subcategories based on a second subset of the characteristic data; triggering the performance of a corrective action associated with the substrate processing system based on one or more of the plurality of regions corresponding to at least one of the plurality of defect subcategories; a processing device for performing A system including:

19. the processing device further comprising: providing the first subset of the characteristic data as an input to a trained machine learning model; obtaining, from the trained machine learning model, output related to predicted data, wherein the processing device identifies the plurality of regions of the substrate corresponding to the first defect category based on the predicted data; and The system of claim 18,

20. The processing device further processes the first subset of the plurality of defect subcategories into: User input indicating one or more subcategories of interest; or Prediction data relating to the output received from the trained machine learning model based on the data input including the historical defect subcategories and the target output including the historical characteristic data.

20. The system of claim 18, wherein the identification is based on one or more of:

Images