Blockage detection using image analysis

Abstract

The method includes identifying an image of a substrate processing equipment component forming a plurality of holes. The method further includes determining, by a processing device, a corresponding adjacent angular distance for each of the plurality of holes and a corresponding area for each of the plurality of holes based on the image. The method further includes identifying, by the processing device, a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding area for each of the plurality of holes. Corrective action associated with the substrate processing equipment component can be performed based on the first subset of the plurality of holes that are at least partially clogged.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to image analysis, and more particularly to blockage detection through image analysis. [Background technology]

[0002] Production equipment includes various components used to manufacture products. For example, substrate processing equipment includes components used to manufacture substrates. The quality and cleanliness of the components affect the performance data of the product. Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to delineate any scope of particular embodiments of the disclosure or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the disclosure, a method includes identifying an image of a substrate processing equipment component forming a plurality of holes. The method further includes determining, by a processing device, a corresponding adjacent angular distance for each of the plurality of holes and a corresponding area for each of the plurality of holes based on the image. The method further includes identifying, by the processing device, a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding area for each of the plurality of holes. Corrective action associated with the substrate processing equipment component can be performed based on the first subset of the plurality of holes that are at least partially clogged.

[0005] In another aspect of the present disclosure, a non-transitory computer-readable storage medium storing instructions that, when executed, cause a processing device to perform operations includes identifying an image of a substrate processing equipment component forming a plurality of holes. The operation further includes determining, based on the image, corresponding adjacent angular distances for each of the plurality of holes and corresponding areas for each of the plurality of holes. The operation further includes identifying a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding areas for each of the plurality of holes. Corrective action associated with the substrate processing equipment component can be performed based on the first subset of the plurality of holes that are at least partially clogged.

[0006] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device can identify an image of a substrate processing equipment component forming a plurality of holes. The processing device can further determine, based on the image, corresponding adjacent angular distances for each of the plurality of holes and corresponding areas for each of the plurality of holes. The processing device can further identify a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding areas for each of the plurality of holes. Corrective action associated with the substrate processing equipment component can be performed based on the first subset of the plurality of holes that are at least partially clogged.

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system configuration, according to certain embodiments. [Figure 2] FIG. 1 illustrates a dataset generator for creating a dataset for a machine learning model, according to certain embodiments. [Figure 3] FIG. 1 is a block diagram illustrating determining predictive data, according to certain embodiments. [Figure 4A] FIG. 1 illustrates an image capture device, in accordance with certain embodiments. [Figure 4B-4C] 1A-1C illustrate images of substrate processing equipment components, according to certain embodiments. [Figures 4D-4F] 1A-1C illustrate hole contours from an image of a substrate processing equipment part, in accordance with certain embodiments. [Figure 4G] 1 illustrates a mapping of nearest holes of a substrate processing equipment component, according to certain embodiments; [Figure 4H] 1 illustrates a mapping of hole helices of a substrate processing equipment piece, according to certain embodiments. [Figures 5A-5D] 1 is a flow diagram of a method relating to blockage detection through image analysis, according to certain embodiments. [Figure 6] FIG. 1 is a block diagram illustrating a computer system, in accordance with certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] SUMMARY Techniques related to clog detection through image analysis (eg, showerhead clog detection from backlit images using image analysis) are described herein.

[0010] Production equipment includes various components used to manufacture products. For example, substrate processing equipment includes components used to manufacture substrates. Some components, such as showerheads, susceptors, etc., form holes (e.g., channels from the top to the bottom) to supply fluids (e.g., gases, liquids, plasma) to portions of the substrate processing system. The quality and cleanliness of these components affect the performance data of the product. For example, fully or partially clogged holes in a component can result in the production of substrates with performance data that do not meet a threshold (e.g., producing defective wafers).

[0011] Some conventional systems manually inspect parts to determine if they meet quality and cleanliness standards, thereby attempting to produce substrates with performance data that meet thresholds. Manual inspection is time-consuming, dependent on the user performing the inspection, and can be inaccurate.

[0012] In some conventional systems, parts undergo cleaning procedures at set intervals to ensure that the parts produce substrates with performance data that meet a threshold. Over time, parts become dirty, damaged, worn, pores become clogged with foreign matter, etc., which are not resolved by conventional cleaning procedures at set intervals. This can result in the production of substrates with performance data that do not meet the threshold, damage to equipment, reduced throughput, production interruptions, etc.

[0013] The devices, systems, and methods disclosed herein provide clog detection through image analysis.

[0014] The processing device identifies an image of a substrate processing equipment component that forms a hole. In some examples, the substrate processing equipment component is a showerhead having an upper surface and a lower surface, which are substantially planar and substantially parallel to each other. A hole (e.g., a channel) may be formed from the upper surface to the lower surface. The hole (e.g., the central axis of the channel) may be oblique (e.g., at an angle of 2 to 8 degrees) relative to the upper and / or lower surfaces. An image (e.g., a backlit image) of one surface (e.g., the upper surface) may be captured while providing light to the opposite surface (e.g., the lower surface) so that light is provided through unclogged holes.

[0015] In some embodiments, the processing device determines, based on the image, a corresponding adjacent angular distance for each of the holes and a corresponding area for each of the holes.

[0016] The processing device identifies a first subset of the at least partially clogged holes based on at least one of the corresponding adjacent angular distances or the corresponding areas of each of the holes, and a corrective action associated with the substrate processing equipment component is to be performed based on the first subset of the at least partially clogged holes.

[0017] Aspects of the present disclosure provide technical advantages. The present disclosure avoids the time, inaccuracy, and subjectivity of traditional manual inspection. The present disclosure produces substrates that meet thresholds, avoids equipment damage, improves throughput, avoids production interruptions, etc.

[0018] Although some embodiments of the present disclosure describe clog detection, the present disclosure may be applied to the detection of partial clogs, worn components, foreign objects, components having cleanliness that does not meet a threshold cleanliness, etc.

[0019] 1 is a block diagram illustrating an example system 100 (an example system configuration) according to certain embodiments. System 100 includes client devices 120, production equipment 124, sensors 126, measurement equipment 128, prediction server 112, and data store 140. In some embodiments, prediction server 112 is part of prediction system 110. In some embodiments, prediction system 110 further includes server machines 170 and 180.

[0020] In some embodiments, one or more of client device 120, production equipment 124, sensors 126, measurement equipment 128, prediction server 112, data store 140, server machine 170, and / or server machine 180 are coupled to one another via network 130 to generate predictive data 160 for performing clog detection. In some embodiments, network 130 is a public network that provides client device 120 with access to prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 with access to production equipment 124, sensors 126, measurement equipment 128, data store 140, and other privately available computing devices. In some embodiments, network 130 includes one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0021] In some embodiments, client device 120 includes a computing device such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet computer, a netbook computer, or the like. In some embodiments, client device 120 includes a corrective action component 122. In some embodiments, corrective action component 122 may also be included in prediction system 110 (e.g., a machine learning processing system). In some embodiments, corrective action component 122 is alternatively included in prediction system 110 (e.g., instead of being included in client device 120). Client device 120 includes an operating system that enables a user to one or more of: integrate, generate, review, or edit data, and provide instructions to a prediction system (e.g., a machine learning processing system).

[0022] In some embodiments, the corrective action component 122 receives user input (e.g., via a graphical user interface (GUI) displayed via the client device 120), receives sensor data 142 from sensors, receives performance data 152 from measurement equipment 128, etc. In some embodiments, the corrective action component 122 transmits data (e.g., user input, sensor data 142, performance data 152, etc.) to the prediction system 110, receives prediction data 160 from the prediction system 110, determines corrective actions based on the prediction data 160, and causes the corrective actions to be implemented. In some embodiments, the corrective action component 122 stores the data (e.g., user input, sensor data 142, performance data 1542, etc.) in the data store 140, and the prediction server 112 retrieves the data from the data store 140. In some embodiments, prediction server 112 stores the output of trained machine learning model 190 (e.g., prediction data 160) in data store 140, and client device 120 retrieves the output from data store 140. In some embodiments, corrective action component 122 receives instructions for corrective actions (e.g., based on prediction data 160) from prediction system 110 and causes the corrective actions to be executed.

[0023] In some embodiments, the predictive data 160 is associated with a corrective action. In some embodiments, the corrective action is associated with one or more of cleaning a substrate processing equipment component, repairing a substrate processing equipment component, replacing a substrate processing equipment component, computational process control (CPC), statistical process control (SPC) (e.g., SPC for comparison to a three-sigma graph, etc.), advanced process control (APC), model-based process control, preventative maintenance, design optimization, updating production parameters, feedback control, machine learning correction, etc. In some embodiments, the corrective action includes providing a warning (e.g., a warning to not use a substrate processing equipment component or production equipment 124 if the predictive data 160 indicates a predicted anomaly, such as a substrate processing equipment component or product anomaly). In some embodiments, the corrective action includes providing feedback control (e.g., cleaning, repairing, and / or replacing a substrate processing equipment component in response to the predictive data 160 indicating a predicted anomaly). In some embodiments, the corrective action includes providing machine learning (e.g., modifying a substrate processing equipment component based on the predictive data 160).

[0024] In some embodiments, prediction server 112, server machine 170, and server machine 180 each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), or the like.

[0025] Prediction server 112 includes a prediction component 114. In some embodiments, prediction component 114 receives sensor data 142 (e.g., received from client device 120 and retrieved from data store 140) and generates predictive data 160 related to clog detection. In some embodiments, prediction component 114 uses one or more trained machine learning models 190 to determine predictive data 160 for clog detection. In some embodiments, trained machine learning models 190 are trained using historical sensor data 144 and historical performance data 154.

[0026] In some embodiments, the prediction system 110 (e.g., prediction server 112, prediction component 114) uses supervised machine learning (e.g., supervised dataset, historical sensor data 144 labeled with historical performance data 154, etc.) to generate the predicted data 160. In some embodiments, the prediction system 110 uses semi-supervised learning (e.g., semi-supervised dataset, performance data 152 is a predictive percentage, etc.) to generate the predicted data 160. In some embodiments, the prediction system 110 uses unsupervised machine learning (e.g., unsupervised dataset, clustering, clustering based on historical sensor data 144, etc.) to generate the predicted data 160.

[0027] In some embodiments, the production equipment 124 (e.g., a cluster tool) is part of a substrate processing system (e.g., an integrated processing system). The production equipment 124 includes one or more of a controller, an enclosure system (e.g., a substrate carrier, a front-opening unified pod (FOUP), an autoteach FOUP, a process kit enclosure system, a substrate enclosure system, a cassette, etc.), a side storage pod (SSP), an aligner device (e.g., an aligner chamber), a factory interface (e.g., a front end of equipment module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robot arm (e.g., disposed in the transfer chamber, disposed in the front interface, etc.), etc. The enclosure system, the SSP, and the load lock are attached to the factory interface, and the robot arm disposed in the factory interface can transfer contents (e.g., substrates, process kit rings, carriers, verification wafers, etc.) between the enclosure system, the SSP, the load lock, and the factory interface. An aligner device is disposed in the factory interface to align the contents. The load locks and processing chambers are attached to a transfer chamber, and a robot arm disposed in the transfer chamber can transfer contents (e.g., substrates, process kit rings, carriers, verification wafers, etc.) between the load locks, processing chambers, and transfer chambers. In some embodiments, the production equipment 124 includes components of a substrate processing system. In some embodiments, the sensor data 142 includes parameters of processes (e.g., etching, heating, cooling, transfer, processing, flow, etc.) performed by components of the production equipment 124. In some embodiments, the substrate processing equipment parts are components of a processing chamber (e.g., showerheads, susceptors, etc.).

[0028] In some embodiments, the sensors 126 provide sensor data 142 (e.g., sensor values, such as historical and current sensor values) related to the production equipment 124. In some embodiments, the sensors 126 include one or more of an imaging sensor (e.g., a camera, an imaging device, etc.), a pressure sensor, a temperature sensor, a flow sensor, a spectroscopic sensor, etc. In some embodiments, the sensor data 142 is used for equipment health and / or product health (e.g., product quality). In some embodiments, the sensor data 142 is received over a period of time.

[0029] In some embodiments, the sensor 126 provides sensor data 142 such as one or more values ​​of image data, leak rate, temperature, pressure, flow rate (e.g., gas flow rate), pump efficiency, spacing (SP), high frequency radio frequency (HFRF), current, power, voltage, etc.

[0030] In some embodiments, sensor data 142 (e.g., historical sensor data 144, current sensor data 146, etc.) is processed (e.g., by client device 120 and / or by prediction server 112). In some embodiments, processing sensor data 142 includes generating features. In some embodiments, the features are patterns of sensor data 142 (e.g., slope, width, height, peak, etc.) or combinations of values ​​from sensor data 142 (e.g., power derived from voltage and current, etc.). In some embodiments, sensor data 142 includes features used by prediction component 114 to obtain prediction data 160.

[0031] In some embodiments, metrology equipment 128 (e.g., imaging equipment, spectroscopic equipment, ellipsometry equipment, etc.) is used to determine metrology data (e.g., inspection data, image data, spectroscopic data, ellipsometry data, material composition, optical, or structural data, etc.) corresponding to substrates produced by production equipment 124 (e.g., substrate processing equipment). In some examples, metrology equipment 128 is used to inspect a portion (e.g., a layer) of a substrate after production equipment 124 processes the substrate. In some embodiments, metrology equipment 128 performs scanning acoustic microscopy (SAM), ultrasound inspection, X-ray inspection, and / or computed tomography (CT) inspection. In some examples, metrology equipment 128 is used to determine the quality of the processed substrate (e.g., layer thickness, layer uniformity, interlayer spacing, etc.) after production equipment 124 deposits one or more layers on the substrate. In some embodiments, metrology equipment 128 includes an imaging device (e.g., SAM equipment, ultrasound equipment, X-ray equipment, CT equipment, etc.). In some embodiments, the performance data 152 includes measurement data from the measurement equipment 128 .

[0032] In some embodiments, data store 140 is memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. In some embodiments, data store 140 includes multiple storage components (e.g., multiple drives or multiple databases) across multiple computing devices (e.g., multiple server computers). In some embodiments, data store 140 stores one or more of sensor data 142, performance data 152, and / or forecast data 160.

[0033] Sensor data 142 includes historical sensor data 144 and current sensor data 146. In some embodiments, sensor data 142 may include one or more of image data, pressure data, a pressure range, temperature data, a temperature range, flow rate data, power data, a comparison parameter for comparing inspection data to threshold data, threshold data, cooling rate data, a cooling rate range, etc. In some embodiments, at least a portion of sensor data 142 is from sensor 126.

[0034] The performance data 152 includes historical performance data 154 and current performance data 156. The performance data 152 may include a hole map corresponding to the substrate processing equipment component (e.g., where holes are located on the substrate processing equipment component, where holes are predicted to be located on the substrate processing equipment component). In some examples, the performance data 152 indicates whether a substrate is properly designed, properly manufactured, and / or properly functioning. In some embodiments, at least a portion of the performance data 152 is related to the quality of the substrates produced by the production equipment 124. In some embodiments, at least a portion of the performance data 152 is based on metrology data from the metrology equipment 128 (e.g., historical performance data 154 includes metrology data indicative of properly processed substrates, substrate characteristic data, yield, etc.). In some embodiments, at least a portion of the performance data 152 is based on inspection of the substrate (e.g., current performance data 156 based on actual inspection). In some embodiments, the performance data 152 includes an indication of an absolute value (e.g., a deformation value that falls short of the threshold deformation value by a calculated value, indicating that the bond interface inspection data is below the threshold data by a calculated value) or a relative value (e.g., a deformation that falls short of the threshold deformation by 5%, indicating that the bond interface inspection data is below the threshold data by 5%). In some embodiments, the performance data 152 indicates that a threshold amount of error (e.g., at least 5% error in manufacturing, at least 5% error in flow rate, at least 5% error in deformation, specification limits) has been met.

[0035] In some embodiments, client device 120 provides performance data 152 (e.g., product data). In some examples, client device 120 provides (e.g., based on user input) performance data 152 indicating product anomalies (e.g., defective products). In some embodiments, performance data 152 includes the amount of product manufactured that was normal or abnormal (e.g., 98% normal product). In some embodiments, performance data 152 indicates the amount of product being manufactured that is predicted to be normal or abnormal. In some embodiments, performance data 152 includes one or more of the yield of a previous batch of product, the average yield, the predicted yield, the predicted amount of defective or non-defective product, etc. In some examples, in response to a first batch of product having a yield of 98% (e.g., 98% of the product were normal and 2% were abnormal), client device 120 provides performance data 152 indicating that an upcoming batch of product may have a 98% yield.

[0036] In some embodiments, the historical data includes one or more of historical sensor data 144 and / or historical performance data 154 (e.g., at least a portion for training machine learning model 190). The current data includes one or more of current sensor data 146 and / or current performance data 156 (e.g., at least a portion that is input to trained machine learning model 190 after training model 190 using historical data). In some embodiments, the current data is used to retrain trained machine learning model 190.

[0037] In some embodiments, the predictive data 160 may be used to trigger the implementation of corrective actions on substrate processing equipment components.

[0038] Performing metrology on products to determine substrate processing equipment parts that do not meet threshold quality and incorrectly manufactured components (e.g., bonded metal plate structures) is costly in terms of time used, metrology equipment 128 used, energy consumed, bandwidth used to send the metrology data, processor overhead to process the metrology data, etc. By providing sensor data 142 to model 190 and receiving predicted data 160 from model 190, system 100 has the technical advantage of avoiding the costly process of using metrology equipment 128 and scrapping substrates.

[0039] Running a production process using substrate processing equipment parts that result in defective product is costly in time, energy, product, substrate processing equipment parts, and production equipment 124, and the cost of identifying the substrate processing equipment parts, causing defective product, cleaning the substrate processing equipment parts, repairing the substrate processing equipment parts, replacing the substrate processing equipment parts, scrapping the old components, etc. By providing sensor data 142 to model 190, receiving predictive data 160 from model 190, and triggering corrective action based on predictive data 160, system 100 has the technical advantage of avoiding the costs of manufacturing, identifying, and scrapping defective substrates.

[0040] In some embodiments, the prediction system 110 further includes a server machine 170 and a server machine 180. The server machine 170 includes a dataset generator 172 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing the machine learning model 190. The dataset generator 172 has the functionality to collect, compile, reduce, and / or partition data to prepare the data for machine learning. In some embodiments (e.g., for small datasets), partitioning for training and validation (e.g., explicit partitioning) is not used. Repeated cross-validation (e.g., 5-fold cross-validation, leave-one-out cross-validation) may be used during training, in which a given dataset is actually repeatedly partitioned into different training and validation sets during training. A model (e.g., the best model, the model with the highest accuracy, etc.) is automatically selected from the vector of models on the separated combined subset. In some embodiments, the dataset generator 172 may explicitly divide the historical data (e.g., historical sensor data 144 and corresponding historical performance data 154) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In this embodiment, some operations of the dataset generator 172 are described in detail below with respect to Figures 2 and 5A. In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of features (e.g., training features).In some examples, the first set of features corresponds to a first set of types of sensor data (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern in values ​​from the first set of sensors) corresponding to each of the datasets (e.g., a training set, a validation set, and a test set), and the second set of features corresponds to a second set of types of sensor data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the datasets.

[0041] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. In some embodiments, engines (e.g., training engine 182, validation engine 184, selection engine 185, and test engine 186) refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 can train machine learning model 190 using one or more sets of features associated with a training set from dataset generator 172. In some embodiments, training engine 182 generates multiple trained machine learning models 190, each corresponding to a distinct set of parameters (e.g., sensor data 142) and corresponding responses (e.g., performance data 152) of the training set. In some embodiments, multiple models are trained on the same parameters with distinct targets for the purpose of modeling multiple effects. In some examples, a first trained machine learning model was trained using sensor data 142 from all sensors 126 (e.g., sensors 1 through 5), a second trained machine learning model was trained using a first subset of sensor data (e.g., from sensors 1, 2, and 4), and a third trained machine learning model was trained using a second subset of sensor data (e.g., from sensors 1, 3, 4, and 5) that partially overlaps with the first subset of features.

[0042] The validation engine 184 may validate the trained machine learning models 190 using a corresponding set of validation set features from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of training set features is validated using a first set of validation set features. The validation engine 184 determines the accuracy of each of the trained machine learning models 190 based on the corresponding set of validation set features. The validation engine 184 evaluates and flags (e.g., discards) trained machine learning models 190 that have an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may select one or more trained machine learning models 190 that have an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may select the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190.

[0043] The test engine 186 can test the trained machine learning models 190 using a corresponding set of test set features from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of training set features is tested using a first set of test set features. The test engine 186 determines the trained machine learning model 190 with the highest accuracy among all of the trained machine learning models based on the test set.

[0044] In some embodiments, machine learning model 190 (e.g., used for classification) refers to a model artifact created by training engine 182 using a training set that includes data inputs and corresponding target outputs (e.g., correctly classifying a condition or ordinal level for each training input). Patterns in the dataset that map the data inputs to the target outputs (correct classifications or levels) may be found, and machine learning model 190 is provided with a mapping that captures these patterns. In some embodiments, machine learning model 190 uses one or more of Gaussian process regression (GPR), Gaussian process classification (GPC), Bayesian neural networks, neural network Gaussian processes, deep belief networks, Gaussian mixture models, or other probabilistic learning methods. Non-probabilistic methods can also be used, including one or more of support vector machines (SVMs), radial basis functions (RBFs), clustering, nearest neighbor algorithms (k-NNs), linear regression, random forests, neural networks (e.g., artificial neural networks), etc. In some embodiments, machine learning model 190 is a multivariate analysis (MVA) regression model.

[0045] The prediction component 114 provides the current sensor data 146 (e.g., as input) to the trained machine learning model 190 and executes the trained machine learning model 190 (e.g., for the input, to obtain one or more outputs). The prediction component 114 can determine (e.g., extract) predicted data 160 from the trained machine learning model 190 and determine (e.g., extract) uncertainty data that indicates a level of confidence that the predicted data 160 corresponds to the current performance data 156. In some embodiments, the prediction component 114 or the corrective action component 122 uses the uncertainty data (e.g., an uncertainty function, or a retrieval function derived from the uncertainty function) to determine whether to use the predicted data 160 to perform corrective action or whether to further train the model 190.

[0046] For purposes of explanation and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (i.e., previous data, historical sensor data 144, and historical performance data 154) and providing current sensor data 146 to one or more trained probabilistic machine learning models 190 to determine predicted data 160. In other implementations, heuristic or rule-based models are used to determine predicted data 160 (e.g., without using a trained machine learning model). In other implementations, non-probabilistic machine learning models may be used. The prediction component 114 monitors the historical sensor data 144 and the historical performance data 154. In some embodiments, any of the information described with respect to data input 210 in FIG. 2 is monitored or otherwise used in the heuristic or rule-based models.

[0047] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 is provided by fewer machines. For example, in some embodiments, server machines 170 and 180 are combined into a single machine, while in some other embodiments, server machine 170, server machine 180, and prediction server 112 are combined into a single machine. In some embodiments, client device 120 and prediction server 112 are combined into a single machine.

[0048] In general, functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed by prediction server 112 in other embodiments, where appropriate. In addition, functions attributed to particular components may be performed by various or multiple components operating together. For example, in some embodiments, prediction server 112 determines corrective actions based on prediction data 160. In another example, client device 120 determines prediction data 160 based on data received from a trained machine learning model.

[0049] Additionally, the functionality of a particular component may be performed by various or multiple components working together. In some embodiments, one or more of prediction server 112, server machine 170, or server machine 180 are accessed as a service offered to other systems or devices through an appropriate application programming interface (API).

[0050] In some embodiments, a "user" is represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. In some examples, a set of individual users federated as a group of administrators is considered a "user."

[0051] Although embodiments of the present disclosure are discussed with respect to determining predictive data 160 for detecting blockages of substrate processing equipment parts in a production facility (e.g., substrate processing facility), in some embodiments the present disclosure may also be applied to quality detection generally. Embodiments may be applied generally to determining part quality based on different types of data.

[0052] 2 illustrates a dataset generator 272 (e.g., dataset generator 172 of FIG. 1 ) for creating a dataset for a machine learning model (e.g., model 190 of FIG. 1 ), according to certain embodiments. In some embodiments, dataset generator 272 is part of server machine 170 of FIG. 1 . The dataset generated by dataset generator 272 of FIG. 2 can be used to train a machine learning model (e.g., see FIG. 5C ) and trigger the implementation of corrective actions (e.g., see FIG. 5D ).

[0053] A dataset generator 272 (e.g., dataset generator 172 in FIG. 1 ) creates a dataset for a machine learning model (e.g., model 190 in FIG. 1 ). The dataset generator 272 creates the dataset using historical sensor data 244 (e.g., historical sensor data 144 in FIG. 1 ) and historical performance data 254 (e.g., historical performance data 154 in FIG. 1 ). The system 200 in FIG. 2 shows the dataset generator 272, a data input 210, and a target output 220 (e.g., target data).

[0054] In some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs) and one or more target outputs 220 that correspond to the data inputs 210. The dataset further includes mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 are also referred to as “features,” “attributes,” or “information.” In some embodiments, the dataset generator 272 provides the dataset to the training engine 182, the validation engine 184, or the test engine 186, and the dataset is used to train, validate, or test the machine learning model 190. Some embodiments of generating a training set are further described with respect to FIG. 5A .

[0055] In some embodiments, dataset generator 272 generates data input 210 and target output 220. In some embodiments, data input 210 includes one or more sets of historical sensor data 244. In some embodiments, historical sensor data 244 includes one or more of sensor data from one or more types of sensors, combinations of sensor data from one or more types of sensors, patterns from sensor data from one or more types of sensors, etc.

[0056] In some embodiments, the dataset generator 272 generates a first data input corresponding to a first set of historical sensor data 244A for training, validating, or testing a first machine learning model, and the dataset generator 272 generates a second data input corresponding to a second set of historical sensor data 244B for training, validating, or testing a second machine learning model.

[0057] In some embodiments, the dataset generator 272 discretizes (e.g., segments) one or more of the data inputs 210 or the target outputs 220 (e.g., for use in a classification algorithm for a regression problem). Discretizing the data inputs 210 or the target outputs 220 (e.g., segmenting by a sliding window) converts continuous values ​​of variables into discrete values. In some embodiments, the discrete values ​​of the data inputs 210 represent discrete historical sensor data 244 to obtain the target outputs 220 (e.g., discrete historical performance data 254).

[0058] The data inputs 210 and target outputs 220 for training, validating, or testing a machine learning model include information for a particular facility (e.g., for a particular substrate production facility). In some examples, the historical sensor data 244 and the historical performance data 254 are for the same production facility.

[0059] In some embodiments, the information used to train the machine learning model is from a particular type of production equipment 124 in a production facility having particular characteristics, allowing the trained machine learning model to determine an outcome for a particular group of production equipment 124 based on current parameter inputs (e.g., current sensor data 146) associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the machine learning model is for components from more than one production facility, allowing the trained machine learning model to determine an outcome for a component based on inputs from one production facility.

[0060] In some embodiments, after generating the dataset and using the dataset to train, validate, or test the machine learning model 190, the machine learning model 190 is further trained, validated, or tested (e.g., current performance data 156 in FIG. 1 ) or adjusted (e.g., adjusting weights associated with the input data of the machine learning model 190, such as connection weights in a neural network).

[0061] 3 is a block diagram illustrating a system 300 for generating predictive data 360 (e.g., predictive data 160 of FIG. 1 ), according to certain embodiments. System 300 is used to determine predictive data 360 via a trained machine learning model (e.g., model 190 of FIG. 1 ) for clog detection (e.g., for taking corrective action).

[0062] In block 310, the system 300 (e.g., the prediction system 110 of FIG. 1 ) performs data splitting (e.g., via the dataset generator 172 of the server machine 170 of FIG. 1 ) of historical data (e.g., the historical sensor data 344 and the historical performance data 354 for the model 190 of FIG. 1 ) to generate a training set 302, a validation set 304, and a test set 306. In some examples, the training set is 60% of the historical data, the validation set is 20% of the historical data, and the test set is 20% of the historical data. The system 300 generates multiple sets of features for each of the training set, validation set, and test set. In some examples, if the historical data includes features derived from 20 sensors (e.g., sensor 126 in FIG. 1 ) and 100 products (e.g., products each corresponding to sensor data from the 20 sensors), the first set of features would be sensors 1-10, the second set of features would be sensors 11-20, the training set would be products 1-60, the validation set would be products 61-80, and the test set would be products 81-100. In this example, the first set of features for the training set would be parameters from sensors 1-10 for products 1-60.

[0063] At block 312, system 300 performs model training using training set 302 (e.g., via training engine 182 of FIG. 1 ). In some embodiments, system 300 trains multiple models using multiple sets of features in training set 302 (e.g., a first set of features in training set 302, a second set of features in training set 302, etc.). For example, system 300 trains machine learning models to generate a first trained machine learning model using a first set of features in the training set (e.g., sensor data from sensors 1-10 of products 1-60) and generate a second trained machine learning model using a second set of features in the training set (e.g., sensor data from sensors 11-20 of products 1-60). In some embodiments, the first trained machine learning model and the second trained machine learning model are combined to generate a third trained machine learning model (e.g., which, in some embodiments, is a better predictor than either the first or second trained machine learning model alone). In some embodiments, the sets of features used in comparing the models overlap (e.g., a first set of features is sensor data from sensors 1-15, and a second set of features is sensor data from sensors 5-20). In some embodiments, hundreds of models are generated, including models with various permutations of features and combinations of models.

[0064] At block 314, the system 300 performs model validation using the validation set 304 (e.g., via the validation engine 184 of FIG. 1 ). The system 300 validates each of the trained models using a corresponding set of features in the validation set 304. For example, the system 300 validates a first trained machine learning model using a first set of features in the validation set (e.g., parameters from sensors 1-10 of products 61-80) and validates a second trained machine learning model using a second set of features in the validation set (e.g., parameters from sensors 11-20 of products 61-80). In some embodiments, the system 300 validates hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 determines the accuracy of each of the one or more trained models (e.g., via model validation) and determines whether one or more of the trained models have an accuracy that meets a threshold accuracy. In response to a determination that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312, where the system 300 performs model training using a different set of features from the training set. In response to a determination that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. The system 300 discards trained machine learning models that have an accuracy that is less than the threshold accuracy (e.g., based on a validation set).

[0065] In block 316, the system 300 performs model selection (e.g., via selection engine 185 of FIG. 1 ) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., selected model 308 based on validation in block 314). In response to a determination that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow returns to block 312, and the system 300 performs model training using a further refined training set corresponding to the further refined set of features to determine the trained model with the highest accuracy.

[0066] At block 318, the system 300 performs model testing (e.g., via test engine 186 of FIG. 1 ) using the test set 306 to test the selected model 308. The system 300 tests the first trained machine learning model using a first set of features in the test set (e.g., sensor data from sensors 1-10 of products 81-100) and determines that the first trained machine learning model meets a threshold accuracy (e.g., based on the first set of features in the test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 overfits the training set 302 and / or the validation set 304 and cannot be applied to other datasets, such as the test set 306), flow proceeds to block 312, where the system 300 performs model training (e.g., retraining) using a different training set corresponding to a different set of features (e.g., sensor data from a different sensor). In response to a determination that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model learns patterns in past data to make predictions, and in block 318, the system 300 applies the model to the remaining data (e.g., the test set 306) to test the predictions.

[0067] In block 320, the system 300 receives current sensor data 346 (e.g., current sensor data 146 of FIG. 1 ) using a trained model (e.g., selected model 308) and determines (e.g., extracts) from the trained model predictive data 360 (e.g., predictive data 160 of FIG. 1 ) for clog detection and performs corrective action. In some embodiments, the current sensor data 346 corresponds to the same feature types in the historical sensor data 344. In some embodiments, the current sensor data 346 corresponds to the same feature types as a subset of the feature types in the historical sensor data 344 used to train the selected model 308.

[0068] In some embodiments, current data is received. In some embodiments, the current data includes current performance data 356 (e.g., current performance data 156 of FIG. 1 ) and / or current sensor data 346. In some embodiments, at least a portion of the current data is received from a measurement device (e.g., measurement device 128 of FIG. 1 ) or via user input. In some embodiments, model 308 is retrained based on the current data. In some embodiments, a new model is trained based on current performance data 356 and current sensor data 346.

[0069] In some embodiments, one or more of blocks 310-320 are performed in various orders and / or with other operations not presented and described herein. In some embodiments, one or more of blocks 310-320 are not performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, and / or model testing of block 318 are not performed.

[0070] FIG. 4A illustrates an image capture device 400 (eg, configured to provide a backlit image of a substrate processing equipment component) according to certain embodiments.

[0071] The image capture device may include a housing 410 including walls (e.g., side walls, a top wall, a bottom wall) that at least partially enclose an interior volume. The housing (e.g., the top wall) may form an opening 412. A light emitting device 414 may be disposed in the interior volume of the housing 410. A spacer 416 may be disposed on or integrated into the light emitting device 414. A substrate processing equipment component 420 that forms a hole 422 may be disposed on the spacer 416.

[0072] The substrate processing equipment component 420 can be a showerhead, a susceptor, etc. In some embodiments, the substrate processing equipment component 420 is cylindrical (e.g., circular periphery). In some embodiments, the holes 422 are perpendicular (e.g., 90 degrees) to the top and / or bottom surfaces of the substrate processing equipment component 420. In some embodiments, the holes 422 are substantially perpendicular (e.g., 90-95 degrees, 90-100 degrees, 95-100 degrees, etc.) to the top and / or bottom surfaces of the substrate processing equipment component 420. In some embodiments, the holes 422 are not perpendicular (e.g., 91-120 degrees, 91-135 degrees, 91-179 degrees, etc.) to the top and / or bottom surfaces of the substrate processing equipment component 420. In some embodiments, the holes 422 are linear channels from the top to the bottom surfaces of the substrate processing equipment component 420. In some embodiments, the holes 422 allow light to project through the substrate processing equipment component 420.

[0073] A sensor 418 (e.g., an imaging device, camera, etc.) may be disposed on the housing 410 above the opening 412. The sensor 418 may capture sensor data (e.g., image data, images, video) of the substrate processing equipment component 420 while the light-emitting device 414 is emitting light. The sensor data may include images of light passing through one or more of the holes 422 (e.g., passing through holes that are at least partially unobstructed). A component (e.g., a diffuser sheet) may be disposed between the sensor 418 and the substrate processing equipment component 420.

[0074] Image capture device 400 can be a darkroom fixture (e.g., a closed, light-tight housing 410) with a pre-mounted camera (e.g., a sensor 418) installed. Sensor 418 can capture high-resolution digital images by passing confined light (e.g., from light-emitting device 414) through a diffuser sheet.

[0075] 4B-4C illustrate processing of an image of a substrate processing equipment component 420, according to certain embodiments. In some embodiments, the substrate processing equipment component 420 forms holes 422 arranged in a spiral, such as that shown in FIGS. 4B-4C. In some embodiments, the substrate processing equipment component 420 forms holes 422 arranged in one or more other patterns (e.g., non-spiral patterns).

[0076] The processing device may receive image 430A (e.g., an original input image, sensor data from sensor 418). For example, the processing device may read the input image with an image processing algorithm. The processing device may process image 430A of FIG. 4B to generate processed image 430B of FIG. 4C. The processing device may mask relevant areas (e.g., areas of holes 422) from image 430A to highlight the showerhead holes and limit (e.g., remove) background error in processed image 430B. The processing device may resize image 430A to a predetermined size to remove the scale factor, thereby generating processed image 430B. Processed image 430B may highlight features of interest (e.g., light passing through holes 422).

[0077] Referring to FIG. 4D, the processing device may further perform contour detection (e.g., by applying a contour detection algorithm) to detect the projection of each illuminated hole (e.g., hole 422) and locate the image foreground, thereby generating processed image 430C from processed image 430B or image 430A.

[0078] To find distinct features related to the contours, the processing device can calculate the centroid and area of ​​each contour. To locate each hole 422 in the substrate processing equipment part 420 (e.g., showerhead), the processing device can derive the associated x- and y-coordinates. The processing device can calculate the centroid of the substrate processing equipment part 420 (e.g., showerhead) based on the centroid distribution of the identified holes 422.

[0079] FIG. 4E shows a close-up of the processed image 430C illustrating the contour detection of individual holes 422 by the transformation.

[0080] FIG. 4F shows a magnified view 430C of the processed image showing the radial distance (r) and angular distance (theta) of the hole 422 and its neighbors (e.g., four neighbors). The processing device can determine the center of gravity of the complete substrate processing equipment part (e.g., showerhead) relative to the hole 422 and convert the Cartesian coordinate system to polar coordinates (radius (r), angle (theta)). The processing device can apply radial magnification to the radial distance of each hole 422 to clearly separate right-handed and left-handed spirals (e.g., Fermat's pattern). The processing device can determine nearest neighbors (e.g., determine four nearest neighbors). The processing device can determine nearest neighbors by using a k-dimensional tree (KDTree) (e.g., a space-partitioning data structure for organizing points in k-dimensional space, a binary space-partitioning tree) or a cKDTree (e.g., a KDTree implemented in C++ and / or a KDTree wrapped in Cython). The processing device can calculate the radial and angular distances of each point with its recognized neighbors. The processing device can use nearest neighbor distances to connect and associate each hole 422 with its assigned spiral (e.g., clockwise or counterclockwise). The processing device can tag the sequence of holes as identifying coordinates (e.g., R1-1, R1-2, R2-849, R2-848, where R1 is the first spiral and R2 is the second spiral).

[0081] 4G shows a graph 440 in which the centers of all connected nearest neighbor holes fall on similar curves forming a right-handed spiral 442 and a left-handed spiral 444 configuration of holes 422. The x-axis is the radial angle (e.g., theta) and the y-axis is the radius.

[0082] 4H shows a graph 450 of holes 422 in a substrate processing equipment component 420. The x-axis is distance from the center in the y-direction, and the y-axis is distance from the center in the x-direction. The graph 450 shows the x-distance from the center and the y-distance from the center of each hole 422.

[0083] In some embodiments, for each hole 422, processing logic compares the adjacent angular distance with the median distance of its spiral to identify fully clogged (e.g., missing) or active (e.g., present) holes. Processing logic may apply a threshold to the area of ​​each hole to identify partially clogged holes and generate a clogged condition report. Based on the clogged condition result, processing logic may trigger additional cleaning cycles, longer cleaning cycle times, maintenance, repair, replacement, etc. The area and condition of each hole 422 along with cleaning time may indicate the effectiveness of the cleaning.

[0084] In some embodiments, processing logic may receive flow data (e.g., related to the flow of fluid through the substrate processing equipment component 420) and determine whether the substrate processing equipment component 420 meets a threshold value (e.g., has been adequately cleaned).

[0085] 5A-5D are flow diagrams of methods 500A-500D relating to clog detection through image analysis, according to certain embodiments. In some embodiments, methods 500A-500D are performed by processing logic including hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 500A-500D are performed at least in part by prediction system 110. In some embodiments, method 500A is performed at least in part by prediction system 110 (e.g., server machine 170 and dataset generator 172 of FIG. 1 , dataset generator 272 of FIG. 2 ). In some embodiments, prediction system 110 uses method 500A to generate a dataset for at least one of training, validating, or testing a machine learning model. In some embodiments, method 500B is performed by client device 120 or prediction system 110 (e.g., corrective action component 122). In some embodiments, method 500C is performed by server machine 180 (e.g., training engine 182, etc.). In some embodiments, method 500D is performed by prediction server 112 (e.g., prediction component 114). In some embodiments, method 500D is performed by client device 120 (e.g., corrective action component 122). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, client device 120, etc.), cause the processing device to perform one or more of methods 500A-500D.

[0086] For ease of explanation, methods 500A-500D are illustrated and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or simultaneously, with other operations not shown and described herein. Furthermore, in some embodiments, not all illustrated operations are performed to implement methods 500A-500D in accordance with the subject matter of the present disclosure. Additionally, those skilled in the art will understand and appreciate that methods 500A-500D may alternatively be represented as a series of interrelated states via a state diagram or events.

[0087] FIG. 5A is a flow diagram of a method 500A for generating a dataset for a machine learning model for generating predicted data (eg, predicted data 160 of FIG. 1) according to certain embodiments.

[0088] Referring to FIG. 5A, in some embodiments, at block 502, processing logic performing method 500A initializes a training set T to an empty set.

[0089] At block 504, processing logic generates a first data input (e.g., a first training input, a first validation input) that includes sensor data (e.g., historical sensor data 144 of FIG. 1 , historical sensor data 244 of FIG. 2 , etc.). In some embodiments, the first data input includes a first set of features related to the type of sensor data, and the second data input includes a second set of features related to the type of sensor data (e.g., as described with respect to FIG. 2 ).

[0090] At block 506, processing logic generates a first target output for one or more of the data inputs (e.g., a first data input). In some embodiments, the first target output is historical performance data (e.g., historical performance data 154 of FIG. 1, historical performance data 254 of FIG. 2).

[0091] At block 508, processing logic optionally generates mapping data indicating an input / output mapping. The input / output mapping (or mapping data) refers to a data input (e.g., one or more of the data inputs described herein), a target output for the data input (e.g., where the target output identifies historical performance data 154), and an association between the data input and the target output.

[0092] At block 510, processing logic adds the mapping data generated at block 508 to the dataset T.

[0093] At block 512, processing logic branches based on whether dataset T is sufficient for at least one of training, validation, and / or testing of machine learning model 190 (e.g., whether the uncertainty of the trained machine learning model meets a threshold uncertainty). If so, execution proceeds to block 514; if not, execution returns to block 504. Note that in some embodiments, the sufficiency of dataset T is determined solely based on the number of input / output mappings in the dataset, while in some other implementations, the sufficiency of dataset T is determined based on one or more other criteria (e.g., a measure of diversity of the data examples, accuracy, etc.) in addition to or instead of the number of input / output mappings.

[0094] At block 514, processing logic provides dataset T (e.g., to server machine 180) to train, validate, and / or test machine learning model 190. In some embodiments, dataset T is a training set and is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a validation set and is provided to validation engine 184 of server machine 180 to perform validation. In some embodiments, dataset T is a test set and is provided to test engine 186 of server machine 180 to perform testing. In the case of a neural network, for example, input values ​​(e.g., numerical values ​​associated with data inputs 210) of a given input / output mapping are input to the neural network, and output values ​​(e.g., numerical values ​​associated with target outputs 220) of the input / output mapping are stored in output nodes of the neural network. The connection weights of the neural network are then adjusted according to a learning algorithm (e.g., backpropagation, etc.), and this procedure is repeated for other input / output mappings of dataset T.

[0095] After block 514, the machine learning model (e.g., machine learning model 190) may be at least one of trained using training engine 182 of server machine 180, validated using validation engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained machine learning model is implemented by prediction component 114 (of prediction server 112) to generate predictive data (e.g., predictive data 160) for clog detection to trigger the execution of corrective actions.

[0096] 5B is a method 500B relating to clog detection through image analysis, according to certain embodiments. In some embodiments, method 500B is performed before and after cleaning of substrate processing equipment parts.

[0097] At block 520 of method 500B, processing logic identifies an image (e.g., a backlit image) of a substrate processing equipment part that forms a hole. The substrate processing equipment part (e.g., substrate processing equipment part 420 of FIG. 4A) can be a showerhead, a susceptor, etc.

[0098] In some embodiments, processing logic receives an image from an image capture device (e.g., image capture device 400 of FIG. 4A). The image capture device can provide light through a substrate processing equipment component. Light that passes through unclogged holes can be shown in the image. Each unclogged hole can be associated with a light shape having a corresponding area. Partially clogged holes are associated with an area that is smaller than the area of ​​less clogged holes (e.g., unclogged holes).

[0099] In some embodiments, an image of a first surface of the substrate processing equipment component is captured while providing light toward a second surface of the substrate processing equipment component that is opposite (e.g., substantially parallel to) the first surface, the light passing through at least a subset of the holes in the substrate processing equipment component.

[0100] In some embodiments, processing logic masks areas from the image to improve visibility of holes formed by substrate processing equipment parts and to reduce background errors, hi some embodiments, processing logic resizes the image to a predetermined size to remove a scale factor.

[0101] In some embodiments, processing logic applies a threshold pixel value to the image. In some embodiments, the pixel format of the image is a byte image, and pixel values ​​are numbers stored as 8-bit integers providing a range of possible values ​​from 0 to 255, where 0 is black and 255 is white. The processing logic can convert all pixel values ​​above the threshold pixel value (e.g., 200) to white (e.g., a pixel value of 255) and can convert all pixel values ​​below the threshold pixel value (e.g., 200) to black (e.g., a pixel value of 0). This can remove gray pixel values ​​from the image.

[0102] In some embodiments, processing logic performs contour detection to detect the projection of each of the holes and locate the image foreground. Processing logic may determine a corresponding centroid and a corresponding area of ​​each of the holes. Processing logic may determine a corresponding Cartesian coordinate of each of the holes. Processing logic may determine a part centroid of the substrate processing equipment part based on the corresponding centroid of each of the holes. Processing logic may convert the corresponding Cartesian coordinate of each of the holes to corresponding polar coordinates based on the part centroid. Processing logic may apply a radial expansion of the corresponding radial distance of the corresponding polar coordinate of each of the holes to separate right-handed and left-handed hole spirals.

[0103] At block 522, processing logic determines, based on the image, the corresponding adjacent angular distances for each of the holes and the corresponding areas for each of the holes.

[0104] At block 524, processing logic identifies a first subset of holes that are at least partially clogged (eg, based on at least one of the corresponding adjacent angular distances or the corresponding areas of each of the holes).

[0105] In some embodiments, to identify the first subset of holes, the processing logic identifies corresponding hole neighbors and corresponding polar coordinates for each of the holes, associates each of the holes with a corresponding hole helix using a corresponding nearest neighbor distance, and compares the corresponding adjacent angular distance for each of the holes to a threshold adjacent angular distance to identify clogged holes (e.g., if the distance between certain adjacent holes is greater than the median distance between adjacent holes, then the holes are likely to be clogged between certain adjacent holes).

[0106] In some embodiments, to identify the first subset of holes, the processing logic compares the corresponding area of ​​each of the plurality of holes to a threshold area (e.g., 50%, 60%, 70%, etc.) to identify partially clogged holes.

[0107] In some embodiments, processing logic may use a machine learning model to determine the threshold area (see, for example, Figures 5C-5D).

[0108] In response to at least a subset of the holes being at least partially clogged at block 526, flow proceeds to block 528. In response to at least a subset of the holes being at least partially unclogging at block 526, flow ends.

[0109] At block 528, processing logic causes corrective action to be taken associated with the substrate processing equipment component based on at least a subset of the holes that are at least partially clogged, and flow returns to block 520. The corrective action may include providing a warning, triggering cleaning action, triggering repair action, triggering replacement, determining the predicted end of life of the substrate processing equipment component, etc.

[0110] Blocks 520-528 may be repeated until no holes are at least partially logged. In some embodiments, processing logic may predict the end of life of a substrate processing equipment component based on the number of corrective actions (e.g., cleaning cycles) taken until the substrate processing equipment component is free of at least partially clogged holes.

[0111] In some embodiments, instead of or in addition to identifying a subset of the pores that are at least partially clogged, the processing logic determines that a foreign object is clogged in one or more of the pores, determines that one or more of the pores are enlarged, etc.

[0112] FIG. 5C is a method for training a machine learning model (e.g., model 190 of FIG. 1) to determine predictive data (e.g., predictive data 160 of FIG. 1) for clog detection by image analysis.

[0113] Referring to FIG. 5C, at block 540 of method 500C, processing logic identifies past sensor data (eg, past sensor data 144 of FIG. 1, past input sensor data).

[0114] At block 542, processing logic identifies historical performance data (e.g., historical performance data 154 of FIG. 1, historical output performance data). At least a portion of the historical sensor data and historical performance data may be associated with a new substrate processing equipment part (e.g., used for benchmarking).

[0115] At block 544, processing logic trains a machine learning model using data inputs including historical sensor data and target outputs including historical performance data to generate a trained machine learning model.

[0116] In some embodiments, the historical sensor data of block 540 includes historical images of the substrate processing equipment component, and the historical performance data of block 542 corresponds to historical substrate processing equipment components. The historical performance data may be associated with substrate quality, such as substrate metrology data, substrate throughput, substrate defects, etc. The historical performance data may be associated with quality of the substrate processing equipment component, such as flow test data (e.g., flowing gas through holes in the substrate processing equipment component), metrology data of the substrate processing equipment component, time to failure of the substrate processing equipment component, etc. In block 544, a machine learning model may be trained using data inputs including the historical images and target outputs including the historical performance data to generate a trained machine learning model configured to identify a threshold area based on an image (e.g., the image of block 520 of FIG. 5B ). In block 524 of FIG. 5B , processing logic may use the threshold area identified via the trained machine learning model to identify a hole that is at least partially clogged (e.g., compare the area of ​​the hole determined in block 522 to the threshold area determined via the trained machine learning model).

[0117] In some embodiments, the historical sensor data of block 540 includes historical images of the substrate processing equipment component, and the historical performance data of block 542 includes historical hole maps corresponding to the historical substrate processing equipment component. In block 544, a machine learning model may be trained using a data input including the historical images and a target output including the historical hole maps to generate a trained machine learning model configured to predict performance data (e.g., performance data of the substrate processing equipment component) based on an image (e.g., the image of block 520 of FIG. 5B ). In response to the predicted performance data satisfying a first threshold, processing logic may trigger a corrective action (e.g., cleaning, repairing, or replacing the substrate processing equipment component). In response to the predicted performance data satisfying a second threshold, processing logic may cause the substrate processing equipment component to be used in the substrate processing system.

[0118] FIG. 5D is a method 500D for triggering the implementation of corrective action using a trained machine learning model for clog detection (e.g., model 190 of FIG. 1).

[0119] 5D, at block 560 of method 500D, processing logic identifies sensor data. In some embodiments, the sensor data at block 540 includes images of substrate processing equipment components.

[0120] At block 562, processing logic provides the sensor data as data input to a trained machine learning model (e.g., trained via block 544 of FIG. 5C).

[0121] At block 564, processing logic receives output from the trained machine learning model related to the predicted data.

[0122] At block 566, processing logic causes the execution of corrective actions based on the predictive data.

[0123] In some embodiments, the sensor data is images of the substrate processing equipment component, and the trained machine learning model of block 562 was trained using a data input including historical images of the substrate processing equipment component and a target output including historical performance data (e.g., substrate quality using the substrate processing equipment component in the past). The predicted data of block 564 may be associated with a threshold area. In some embodiments, in block 566, processing logic compares the hole area of ​​block 522 of FIG. 5B to a threshold area to identify holes that are at least partially clogged in block 524 of FIG. 5B and then triggers corrective action (e.g., for the hole area to meet the threshold area) to clean, repair, or replace the substrate processing equipment component (e.g., see block 528 of FIG. 5B).

[0124] In some embodiments, the sensor data is images of the substrate processing equipment component, and the trained machine learning model of block 562 was trained using a data input including historical images of the substrate processing equipment component and a target output including historical performance data, including historical hole maps corresponding to the substrate processing equipment component. The predicted data of block 564 may be associated with predicted performance data (e.g., performance data of the substrate processing equipment component) based on the images (e.g., the images of block 520 of FIG. 5B ). In response to the predicted performance data satisfying a first threshold, the processing logic may trigger a corrective action (e.g., cleaning, repairing, or replacing the substrate processing equipment component). In response to the predicted performance data satisfying a second threshold, the processing logic may cause the substrate processing equipment component to be used in the substrate processing system.

[0125] 6 is a block diagram illustrating a computer system 600, according to certain embodiments. In some embodiments, computer system 600 is one or more of client device 120, prediction system 110, server machine 170, server machine 180, prediction server 112, or image capture device 400.

[0126] In some embodiments, computer system 600 is connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). In some embodiments, computer system 600 operates as a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. In some embodiments, computer system 600 is provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Furthermore, the term "computer" is intended to include any collection of computers that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methods described herein.

[0127] In a further aspect, computer system 600 includes a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 616, which communicate with each other via a bus 608.

[0128] In some embodiments, the processing device 602 is provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIM) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of instruction set types), or a special-purpose processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0129] In some embodiments, computer system 600 further includes a network interface device 622 (e.g., coupled to a network 674). In some embodiments, computer system 600 further includes a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generating device 620.

[0130] In some implementations, the data storage device 616 includes a non-transitory computer-readable storage medium 624 that stores instructions 626 encoding any one or more of the methods or functions described herein, including instructions encoding the components of FIG. 1 (e.g., the corrective action component 122, the prediction component 114, etc.) and instructions for performing the methods described herein (e.g., one or more of methods 500A-500D).

[0131] In some embodiments, the instructions 626 also reside, completely or partially, within the volatile memory 604 and / or within the processing device 602 during execution by the computer system 600; and thus, in some embodiments, the volatile memory 604 and the processing device 602 also constitute machine-readable storage media.

[0132] Although computer-readable storage medium 624 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that can store or encode a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0133] In some embodiments, the methods, components, and features described herein are implemented by discrete hardware components or are integrated into the functionality of other hardware components, such as an ASIC, FPGA, DSP, or similar device. In some embodiments, the methods, components, and features are implemented by firmware modules or functional circuitry within a hardware device. In some embodiments, the methods, components, and features are implemented in any combination of hardware devices and computer program components, or in a computer program.

[0134] Unless otherwise specified, terms such as "identify," "determine," "mask," "resize," "execute," "convert," "apply," "associate," "compare," "train," "cause," "receive," "provide," "obtain," "update," and the like refer to actions and processes performed or implemented by a computer system that manipulate data represented as physical (electronic) quantities in computer system registers and memory and convert it into other data similarly represented as physical quantities in computer system memory or registers, or other such information storage, transmission, or display devices. In some embodiments, the terms "first," "second," "third," "fourth," and the like, as used herein, are meant as labels to distinguish between different elements and do not have any ordering meaning due to their numerical symbolic designation.

[0135] The examples described herein also relate to apparatus for performing the methods described herein. In some embodiments, the apparatus comprises a general-purpose computer system that is specially constructed to perform the methods described herein or that is selectively programmed by a computer program stored on the computer system. Such a computer program is stored on a computer-readable tangible storage medium.

[0136] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. In some embodiments, various general-purpose systems are used in accordance with the teachings described herein. In some embodiments, more specialized apparatus are constructed to perform the methods described herein and / or each of their individual functions, routines, subroutines, or operations. Examples of structures for various of these systems are set forth in the description above.

[0137] The above description is illustrative, and not limiting. While the present disclosure has been described with reference to particular illustrative examples and embodiments, it will be recognized that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. identifying an image of a substrate processing equipment component that defines a plurality of holes; determining, by a processing device, a corresponding adjacent angular distance for each of the plurality of holes and a corresponding area for each of the plurality of holes based on the image; identifying, by the processing device, a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding areas of each of the plurality of holes, wherein corrective action associated with the substrate processing equipment component may be taken based on the first subset of the plurality of holes that are at least partially clogged; A method comprising:

2. 10. The method of claim 1, wherein the image of the first surface of the substrate processing equipment component is captured while providing light toward a second surface of the substrate processing equipment component opposite the first surface, the light passing through at least a second subset of the plurality of holes.

3. masking areas from the image to improve visibility of the holes formed by the substrate processing equipment components and reduce background error; Resizing the image to a predetermined size to remove the scale factor; The method of claim 1 further comprising:

4. performing contour detection to detect a projection of each of the plurality of holes and locate an image foreground; determining a corresponding centroid and the corresponding area of ​​each of the plurality of holes; determining corresponding Cartesian coordinates for each of the plurality of holes; determining a component center of gravity of the substrate processing equipment component based on the corresponding center of gravity of each of the plurality of holes; converting the corresponding Cartesian coordinates of each of the plurality of holes to corresponding polar coordinates based on the part center of gravity; applying a radial expansion of the corresponding radial distances of the corresponding polar coordinates of each of the plurality of holes to separate right-handed and left-handed hole spirals; The method of claim 1 further comprising:

5. identifying the first subset of the plurality of holes that are at least partially clogged; identifying a corresponding hole neighborhood and corresponding polar coordinates for each of the plurality of holes; associating each of the plurality of holes with a corresponding hole spiral using a corresponding nearest neighbor distance; comparing the corresponding adjacent angular distance of each of the plurality of holes to a threshold adjacent angular distance to identify clogged holes; The method of claim 1 , comprising:

6. 2. The method of claim 1, wherein identifying the first subset of the plurality of holes that are at least partially clogged comprises comparing the corresponding area of ​​each of the plurality of holes to a threshold area to identify partially clogged holes.

7. identifying a historical image of a historical substrate processing equipment component; Identifying historical performance data corresponding to historical substrate processing equipment components; training a machine learning model using a data input including the past images and a target output including the past performance data to generate a trained machine learning model configured to identify the threshold area; The method of claim 6 further comprising:

8. identifying a historical image of a historical substrate processing equipment component; identifying a historical hole map corresponding to the historical substrate processing equipment part; training a machine learning model using a data input including the historical images and a target output including the historical hole maps to generate a trained machine learning model configured to predict performance data based on the images; The method of claim 7 further comprising:

9. A non-transitory computer-readable storage medium storing instructions that, when executed, identifying an image of a substrate processing equipment component that defines a plurality of holes; determining a corresponding adjacent angular distance for each of the plurality of holes and a corresponding area for each of the plurality of holes based on the image; identifying a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding areas of each of the plurality of holes, wherein corrective action associated with the substrate processing equipment component may be taken based on the first subset of the plurality of holes that are at least partially clogged; A non-transitory computer-readable storage medium that causes a processing device to perform operations including:

10. 10. The non-transitory computer-readable storage medium of claim 9, wherein the image of the first surface of the substrate processing equipment component is captured while providing light toward a second surface of the substrate processing equipment component opposite the first surface, the light passing through at least a second subset of the plurality of holes.

11. The operation is masking areas from the image to improve visibility of the holes formed by the substrate processing equipment components and reduce background error; Resizing the image to a predetermined size to remove the scale factor; 10. The non-transitory computer-readable storage medium of claim 9, further comprising:

12. The operation is performing contour detection to detect a projection of each of the plurality of holes and locate an image foreground; determining a corresponding centroid and the corresponding area of ​​each of the plurality of holes; determining corresponding Cartesian coordinates for each of the plurality of holes; determining a component center of gravity of the substrate processing equipment component based on the corresponding center of gravity of each of the plurality of holes; converting the corresponding Cartesian coordinates of each of the plurality of holes to corresponding polar coordinates based on the part center of gravity; applying a radial expansion of the corresponding radial distances of the corresponding polar coordinates of each of the plurality of holes to separate right-handed and left-handed hole spirals; 10. The non-transitory computer-readable storage medium of claim 9, further comprising:

13. identifying the first subset of the plurality of holes that are at least partially clogged; identifying a corresponding hole neighborhood and corresponding polar coordinates for each of the plurality of holes; associating each of the plurality of holes with a corresponding hole spiral using a corresponding nearest neighbor distance; comparing the corresponding adjacent angular distance of each of the plurality of holes to a threshold adjacent angular distance to identify clogged holes; 10. The non-transitory computer-readable storage medium of claim 9, comprising:

14. 10. The non-transitory computer-readable storage medium of claim 9, wherein identifying the first subset of the plurality of holes that are at least partially clogged comprises comparing the corresponding area of ​​each of the plurality of holes to a threshold area to identify partially clogged holes.

15. The operation is identifying a historical image of a historical substrate processing equipment component; Identifying historical performance data corresponding to historical substrate processing equipment components; training a machine learning model using a data input including the past images and a target output including the past performance data to generate a trained machine learning model configured to identify the threshold area; 15. The non-transitory computer-readable storage medium of claim 14, further comprising:

16. The operation is identifying a historical image of a historical substrate processing equipment component; identifying a historical hole map corresponding to the historical substrate processing equipment part; training a machine learning model using a data input including the historical images and a target output including the historical hole maps to generate a trained machine learning model configured to predict performance data based on the images; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

17. Memory and a processing device coupled to the memory, the processing device comprising: identifying an image of a substrate processing equipment component that defines a plurality of holes; determining a corresponding adjacent angular distance for each of the plurality of holes and a corresponding area for each of the plurality of holes based on the image; identifying a first subset of the plurality of holes that are at least partially clogged based on at least one of the corresponding adjacent angular distances or the corresponding areas of each of the plurality of holes, wherein corrective action associated with the substrate processing equipment component may be taken based on the first subset of the plurality of holes that are at least partially clogged; a processing device for performing Including, the system.

18. 20. The system of claim 17, wherein the image of the first surface of the substrate processing equipment component is captured while providing light toward a second surface of the substrate processing equipment component opposite the first surface, the light passing through at least a second subset of the plurality of holes.

19. the processing device: masking areas from the image to improve visibility of the holes formed by the substrate processing equipment components and reduce background error; Resizing the image to a predetermined size to remove the scale factor; The system of claim 17 further comprising:

20. the processing device: performing contour detection to detect a projection of each of the plurality of holes and locate an image foreground; determining a corresponding centroid and the corresponding area of ​​each of the plurality of holes; determining corresponding Cartesian coordinates for each of the plurality of holes; determining a component center of gravity of the substrate processing equipment component based on the corresponding center of gravity of each of the plurality of holes; converting the corresponding Cartesian coordinates of each of the plurality of holes to corresponding polar coordinates based on the part center of gravity; applying a radial expansion of the corresponding radial distances of the corresponding polar coordinates of each of the plurality of holes to separate right-handed and left-handed hole spirals; The system of claim 17 further comprising:

Images