Method, system, and computer-readable medium for part, sensor, and metrology data integration

Abstract

A method includes receiving component data associated with a corresponding component of a substrate processing apparatus, sensor data associated with one or more corresponding substrate processing operations performed by the substrate processing apparatus to generate one or more corresponding substrates, and metrological data associated with one or more corresponding substrates generated by one or more corresponding substrate processing operations performed by the substrate processing apparatus including the corresponding component. The method further includes generating an aggregated set of component sensor metrological data, including a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of metrological data. The method further includes causing analysis of the aggregated set of component sensor metrological data to generate one or more outputs for performing a calibration action associated with a corresponding component of the substrate processing apparatus.

Description

Technical Field

[0001] This case concerns data integration, and more specifically, the integration of components, sensors, and measurement data. Prior Technology

[0002] Products are produced by performing one or more manufacturing processes using manufacturing equipment. Manufacturing equipment is made of multiple components. Defective products can result from suboptimal components and suboptimal manufacturing processes. Summary of the Invention

[0003] The following is a brief overview of this case to provide a basic understanding of some aspects of it. This overview is not an exhaustive summary of this case. It is not intended to identify any important or key elements of this case, nor is it intended to describe any category of any particular implementation of this case or any category of the scope of the claims. Its sole purpose is to present some concepts of this case in a concise form as a prelude to the more detailed description that follows.

[0004] In one embodiment of this invention, a method includes receiving a plurality of sets of component data, each of the plurality of sets of component data being associated with a corresponding component of a substrate processing apparatus. The method further includes receiving a plurality of sets of sensor data, each of the plurality of sets of sensor data being associated with one or more corresponding substrate processing operations performed by the substrate processing apparatus to generate one or more corresponding substrates. The method further includes receiving a plurality of sets of metrological data, each of the plurality of sets of metrological data being associated with one or more corresponding substrates generated by one or more corresponding substrate processing operations performed by a substrate processing apparatus including a corresponding component. The method further includes generating a plurality of aggregated sets of component sensor metrological data, each of the aggregated sets of component sensor metrological data including a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of metrological data. The method further includes causing analysis of the aggregated sets of component sensor metrological data to generate one or more outputs for performing a correction action associated with a corresponding component of the substrate processing apparatus.

[0005] In one embodiment of this invention, a system includes a memory and a processing device coupled to the memory. The processing device is configured to receive a plurality of sets of component data, each set of component data being associated with a corresponding component of a substrate processing apparatus. The processing device is further configured to receive a plurality of sets of sensor data, each set of sensor data being associated with one or more corresponding substrate processing operations performed by the substrate processing apparatus to generate one or more corresponding substrates. The processing device is further configured to receive a plurality of sets of measurement data, each set of measurement data being associated with one or more corresponding substrates generated by one or more corresponding substrate processing operations performed by the substrate processing apparatus including the corresponding component. The processing device is further configured to generate a plurality of aggregated sets of component sensor measurement data, each set of aggregated component sensor measurement data including a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of measurement data. The processing device is further configured to cause analysis of the aggregated sets of component sensor measurement data to generate one or more outputs for performing correction actions associated with corresponding components of the substrate processing apparatus.

[0006] In one embodiment of this case, a non-transitory computer-readable medium stores instructions thereon that, when executed by a processing device, cause the processing device to receive a plurality of sets of component data, each of the plurality of sets of component data being associated with a corresponding component of a substrate processing apparatus. The processing device is further configured to receive a plurality of sets of sensor data, each of the plurality of sets of sensor data being associated with one or more corresponding substrate processing operations performed by the substrate processing apparatus to generate one or more corresponding substrates. The processing device is further configured to receive a plurality of sets of measurement data, each of the plurality of sets of measurement data being associated with one or more corresponding substrates generated by one or more corresponding substrate processing operations performed by a substrate processing apparatus including corresponding components. The processing device is further configured to generate a plurality of aggregated sets of component sensor measurement data, each of the aggregated sets of component sensor measurement data including a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of measurement data. The processing device is further used to cause analysis of multiple sets of aggregated component sensor measurement data to generate one or more outputs for performing correction actions associated with corresponding components of the substrate processing equipment. Simple Explanation of the Diagram

[0007] This case is illustrated by way of example in the accompanying drawings and is not intended to be limiting.

[0008] Figure 1 is a block diagram illustrating an exemplary system architecture according to certain embodiments.

[0009] Figure 2A is a flowchart of a method for integrating components, sensors, and measurement data according to certain embodiments.

[0010] Figure 2B is a block diagram integrating components, sensors, and measurement data according to certain embodiments.

[0011] Figure 3A illustrates an exemplary dataset generator for creating a dataset for a machine learning model, according to certain embodiments.

[0012] Figure 3B is a block diagram illustrating, according to certain embodiments, the determination of forecast data.

[0013] Figures 4A to 4C are flowcharts of methods associated with integrating components, sensors, and measurement data according to certain embodiments.

[0014] Figure 5 is a block diagram illustrating a computer system according to certain embodiments. Implementation

[0015] This article describes the technologies involved in the integration of components, sensors, and measurement data.

[0016] Manufacturing equipment is used to produce products. For example, substrate processing equipment is used to produce substrates. Substrate processing equipment is made from components (e.g., from component manufacturing equipment). Substrate processing equipment performs substrate processing. It determines the performance data of the substrate. For example, metrology equipment is used to determine the metrological data (e.g., measurements, dimensions, material properties, imaging data, etc.) of the substrate produced by substrate processing equipment through substrate processing operations. Substrates with performance data that do not meet thresholds (e.g., dimensions that do not meet thresholds, etc.) may be the result of one or more suboptimal components of the substrate processing equipment and / or one or more suboptimal parameters (e.g., temperature, pressure, flow rate, etc.) of the substrate processing operation.

[0017] Conventionally, product improvement is a trial-and-error process. In response to substrate performance data failing to meet thresholds, users conventionally replace one or more components of the substrate processing equipment and / or change one or more manufacturing parameters of the substrate processing operation. This process is repeated until the substrate performance data meets the thresholds. This is time-consuming, wasteful of components and substrates, reduces yield, and may not produce substrates with optimal performance.

[0018] Conventionally, component data, sensor data, and measurement data are stored in different locations and in different formats, and may not be easily accessible to the user. This prevents data analysis of different types of data. In some cases, when the cause of poor product performance is actually due to something different, the cause of the poor product performance is incorrectly attributed to one or more of the following: component manufacturing, component assembly, board handling operations, maintenance, etc.

[0019] The system and method disclosed herein are integrated with components, sensors, and metrological data. Component data, sensor data, and metrological data are identified. Component data is associated with corresponding components of the substrate processing equipment. For example, component data may include dimensions, material properties, storage information, transportation information, environmental data, etc. Sensor data is associated with substrate processing operations performed by the substrate processing equipment to produce the substrate. For example, sensor data may include temperature, pressure, flow rate, etc. Metering data is associated with the substrate produced by the substrate processing operations performed by the substrate processing equipment including components. Metering data may include substrate measurements, substrate images, etc.

[0020] Component data, sensor data, and measurement data are aggregated (e.g., linked, integrated, etc.) to form aggregated component sensor measurement data. In some embodiments, each of the component data, sensor data, and measurement data includes an identifier, and a common portion between the identifiers is used to identify component sensor measurement matches to generate an aggregated set of component sensor measurement data.

[0021] This process leads to the analysis of a collection of aggregated component sensor measurement data to generate an output. The output is used to perform correction actions associated with components in a substrate processing apparatus. Correction actions may include updating component design, component quality, component dimensions, component feature layout, component manufacturing operations to produce the component, performing root cause analysis to determine updates to component or substrate processing operations, etc. In some embodiments, analyzing the aggregated component sensor measurement data includes storing the aggregated component sensor measurement data. In some embodiments, analyzing the aggregated component sensor measurement data includes training a machine learning model using data input from the aggregated component sensor measurement data to generate a trained machine learning model. In some embodiments, analyzing the aggregated component sensor measurement data includes providing the aggregated component sensor measurement data as data input to a trained machine learning model to receive an output to perform correction actions associated with components in a substrate processing apparatus.

[0022] Compared to conventional solutions, the present invention offers technological advantages. It determines corrective actions without the need for conventional human trial and error. This reduces time, component waste, product waste, and increases yield. Compared to conventional solutions that consider only a portion of the data, this invention involves analyzing component, sensor, and metrology data to determine corrective actions based on component and substrate manufacturing parameters. Compared to conventional solutions, the use of aggregated component sensor metrology data in this invention results in faster and less error-prone corrective actions. This invention can be used to determine the impact of component variability and / or operational variability on substrate production (e.g., controlling tooling operation to improve on-wafer performance), which may be impossible to determine using conventional solutions. This invention can be used to correlate component manufacturing data with on-wafer performance to identify good and bad components and improve component manufacturing processes to increase the number of components with good on-wafer performance. Compared to conventional solutions, this results in better components, better substrates, and higher yield.

[0023] While some embodiments of this invention describe the integration of component data, sensor data, and measurement data, different combinations of data may be used in some embodiments. For example, aggregated component sensor data, aggregated component measurement data, aggregated sensor measurement data, component manufacturing data aggregated with other data, component mounting data aggregated with other data, and / or the like may be used.

[0024] Although some embodiments of this application describe component data, sensor data, and measurement data associated with substrate processing equipment, in some embodiments, component data, sensor data, and measurement data may correspond to other types of equipment (e.g., manufacturing equipment) used to produce products.

[0025] Figure 1 is a block diagram illustrating an exemplary system architecture of system 100 according to certain embodiments. System 100 includes a data integration server 102, a client device 106, a component system 120, a sensor system 130, a metering system 140, a prediction server 112, and a data storage device 150. The prediction server 112 may be a component of the prediction system 110. The prediction system 110 may further include server machines 170 and 180.

[0026] Component system 120 may include component server 122, component manufacturing equipment 124, sensor 126, and identifier reader 128 (e.g., for reading component numbers and / or component serial numbers). Sensor system 130 includes sensor server 132 (e.g., a field service server (FSS) at a manufacturing facility), substrate processing equipment 134, sensor 136, and identifier reader 138 (e.g., a front-opening unified pod (FOUP) radio frequency identification (RFID) reader for sensor system 130). Metrology system 140 includes metrology server 142 (e.g., metrology database, metrology folder, etc.), metrology equipment 144, and identifier reader 146 (e.g., a FOUP RFID reader for metrology system 140).

[0027] The sensor 126 can provide component data 152 associated with the component produced by the manufacturing equipment 124 (e.g., manufacturing parameters of the component manufacturing equipment 124, component measurements, etc.).

[0028] Component data 152 may include one or more of the following values: temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), voltage, current, flow rate, power, etc. Component data 152 may be associated with or indicate manufacturing parameters, such as hardware parameters (e.g., settings or components of component manufacturing equipment 124 (e.g., size, type, etc.)) or processing parameters of component manufacturing equipment 124. Component data 152 is available during manufacturing operations performed on component manufacturing equipment 124 (e.g., equipment readings when manufacturing a component). Component data 152 may be different for each component. Components may be used in substrate processing equipment 134. For example, components may be spray heads, bases, heaters, rubber diaphragms to apply pressure to a first side of the substrate when performing substrate processing on a second side of the substrate, such as a substrate, electrostatic chuck, etc.

[0029] Sensor 136 can provide sensor data 154 associated with the substrate produced by substrate processing equipment 134 (e.g., manufacturing parameters of substrate processing equipment 134, etc.). Sensor data 154 may include one or more values ​​of: temperature (e.g., heater temperature), spacing (SP), pressure, high-frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, flow rate, power, voltage, etc. Sensor data 154 may be associated with or indicate manufacturing parameters, such as hardware parameters of substrate processing equipment 134 (e.g., settings or components of substrate processing equipment 134 (e.g., size, type, etc.)) or processing parameters. Sensor data 154 can be provided when substrate processing equipment 134 performs substrate processing operations (e.g., equipment readings when processing substrates). Sensor data 154 may be different for each product (e.g., each substrate).

[0030] The metrology equipment 144 can provide metrology data 156 (e.g., measurements, material properties, storage information, environmental conditions, etc.) of the substrate produced by the substrate processing equipment 134. The metrology equipment 144 can provide metrology data 156 associated with the substrate produced by the substrate processing equipment 134 (e.g., substrate property data). The metrology data 156 may include one or more of the following values: film property data (e.g., wafer space film properties), dimensions (e.g., thickness, height, etc.), dielectric constant, dopant concentration, density, defects, etc. The metrology data 156 may be for finished or semi-finished products. The metrology data 156 may be different for each product (e.g., each substrate).

[0031] Identifier readers 128, 138, and / or 146 may provide identifier 158. In some embodiments, identifier readers 128, 138, and / or 138 may be RFID readers (e.g., FOUP RFID readers, carrier RFID readers), imaging devices (e.g., barcode readers, etc.), and / or the like. Identifier 158 includes one or more of the following: indication of a carrier (e.g., carrier identifier), indication of a product (e.g., which substrate in the carrier, substrate order), and indication of a timestamp (e.g., date, time, etc.).

[0032] In some embodiments, the identifier reader 128 of the component system 120 may provide one or more identifiers 158 associated with the manufacture of the component, installation of the component, storage of the component, transportation of the component, etc.

[0033] In some embodiments, the identifier reader 138 of the sensor system 130 (e.g., a FOUP RFID reader) can provide an identifier 158 (e.g., a carrier identifier, such as a FOUP identifier, a substrate carrier identifier, a time slot identifier, etc.).

[0034] In some embodiments, the metering system’s identifier reader 146 (e.g., a FOUP RFID reader) can provide an identifier 158 (e.g., a carrier identifier, such as a FOUP identifier, a substrate carrier identifier, a time slot identifier, etc.).

[0035] In some embodiments, a product carrier (e.g., a FOUP, a wafer carrier) can transfer a substrate from a substrate processing apparatus 134 to a metrology apparatus 144. The substrate can maintain the same order in the sensor system 130 and in the metrology system 140 (e.g., the same position in the FOUP or wafer carrier). For example, the substrate can be loaded into and unloaded from the substrate processing apparatus 134 in the same order as loading and unloading the substrate into and out of the metrology apparatus 144 (e.g., for performing substrate processing and providing sensor data 154 via the sensor server 132) to the same order as loading and unloading the substrate into and out of the metrology apparatus 144 (e.g., for providing metrology data 156 via the metrology system 140). In some embodiments, the identifier 158 provided by the identifier reader 138 (e.g., the FOUP ID associated with the sensor system 130) and the identifier reader 138 provided by the identifier reader 146 (e.g., the FOUP ID associated with the metering system 140) corresponding to the same product are associated with the same product carrier (e.g., the same FOUP) and / or carrier identifier (e.g., the sensor carrier identifier and the metering carrier identifier are the same).

[0036] Data integration server 102, client device 106, component system 120 (e.g., component server 122, component manufacturing equipment 124, sensor 126, identifier reader 128, etc.), sensor system 130 (e.g., sensor server 132, substrate processing equipment 134, sensor 136, identifier reader 138, etc.), metering system 140 (e.g., metering server 142, metering device 144, identifier reader 146, etc.), prediction server 112, data storage 150, server machine 170, and server machine 180 can be coupled to each other via network 116 to generate aggregated component sensor metering data 160 for performing calibration operations. In some embodiments, network 116 is a public network that provides client device 106 with access to prediction server 112, data storage 150, and other publicly available computing devices. In some embodiments, network 116 is a dedicated network that provides data integration server 102 with access to sensor system 130, metering system 140, data storage 150, and other dedicated computing devices, and provides client device 106 with access to prediction server 112, data storage 150, and other dedicated computing devices. Network 116 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0037] Client device 106 may include computing devices such as personal computers (PCs), desktop computers, laptops, mobile phones, smartphones, tablets, laptops, etc. Client device 106 may include a calibration action component 108. Calibration action component 108 may receive user input (e.g., via a graphical user interface (GUI) displayed on client device 106) of instructions associated with substrate processing equipment 134 (e.g., components of the substrate processing equipment, substrates produced by the substrate processing equipment, etc.). In some embodiments, calibration action component 108 sends instructions to prediction system 110, receives output from prediction system 110 (e.g., prediction data 64), determines a calibration action based on the output, and causes the calibration action to be performed. Client device 106 may include an operating system that allows a user to generate, view, or edit data (e.g., instructions associated with components, substrate processing equipment 134, substrates, calibration actions associated with components, etc.).

[0038] Corrective actions may be associated with one or more of the following: computational process control (CPC), statistical process control (SPC), automatic process control (APC), preventive operation and maintenance, design optimization, updating manufacturing parameters, feedback control, machine learning modification, etc.

[0039] In some embodiments, the corrective action provides an alarm (e.g., warning to stop or not perform the manufacturing process if prediction data 164 indicates that a component or product has poor performance, such as predicting that the hole in the component is not round). In some embodiments, the corrective action provides feedback control (e.g., modifying manufacturing parameters, such as slowing down drill removal in response to prediction data 164 indicating that the predicted hole is not round). In some embodiments, the corrective action provides machine learning (e.g., modifying one or more manufacturing parameters, such as drill rotation, insertion rate, removal rate, etc., based on prediction data 164).

[0040] In some embodiments, the calibration action results in the updating of one or more manufacturing parameters. Manufacturing parameters may include hardware parameters (e.g., component replacement, use of certain components, etc.) and / or processing parameters (e.g., temperature, pressure, flow rate, rate, etc.). In some embodiments, the calibration action results in preventative operational maintenance (e.g., replacement, handling, cleaning, etc. of components in component manufacturing equipment 124 and / or substrate processing equipment 134). In some embodiments, the calibration action results in design optimization (e.g., updating manufacturing parameters, manufacturing processes, component design, substrate processing equipment 134, etc., for an optimized product).

[0041] Data integration server 102, prediction server 112, sensor server 132, measurement server 142, server machine 170, and server machine 180 may each include one or more computing devices, such as rack servers, router computers, server computers, PCs, host computers, laptops, tablet computers, desktop computers, graphics processing units (GPUs), application-specific integrated circuits (ASICs) (e.g., tensor processing units (TPUs)), etc.

[0042] Data integration server 102 may include data integration component 104. Data integration component 104 may receive a set of component data 152 (e.g., from component server 122, from data storage 150), a set of sensor data 154 (e.g., from sensor server 132, from data storage 150), and a set of metering data 156 (e.g., from metering server 142, from data storage 150), determine component sensor metering matches among the set of component data 152, the set of sensor data 154, and the set of metering data 156, generate an aggregated set of component sensor metering data 160 based on the sensor metering matches, and store the aggregated set of component sensor metering data 160 in data storage 150.

[0043] Prediction server 112 may include prediction component 114. In some embodiments, prediction component 114 may retrieve aggregated component sensor measurement data 160 from data storage and generate an output (e.g., prediction data 164) for performing correction actions associated with components of substrate processing apparatus 134 based on the aggregated component sensor measurement data 160. In some embodiments, prediction component 114 may use a trained machine learning model 190 to determine the output for performing the correction actions. The trained machine learning model 190 may be trained using the aggregated component sensor measurement data 160 to learn important process and hardware parameters. Generating the output for performing the correction actions by the trained machine learning model 190 may include specifying optimal operating conditions (e.g., processing parameters) and / or space (e.g., hardware parameters) by the trained machine learning model 190. In some embodiments, the prediction component 114 determines the prediction data 164 for performing the correction action by providing current data 162 (e.g., currently aggregated component sensor measurement data 160) to a trained machine learning model 190, obtaining output from the trained machine learning model 190, and determining the prediction data 164 based on the output.

[0044] Data storage 150 may be memory (e.g., random access memory), a drive (e.g., a hard drive, flash drive), a database system, or another type of component or device capable of storing data. Data storage 150 may include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers). Data storage 150 may store component data 152, sensor data 154, measurement data 156, identifier 158, aggregated component sensor measurement data 160, current data 162, prediction data 164 (e.g., predicted performance data), and performance data 166. Component data 152, sensor data 154, measurement data 156, and aggregated component sensor measurement data 160 may include historical data (e.g., data used to train machine learning model 190). The current data 162 may be data (e.g., sensor data 154, measurement data 156, and / or aggregated component sensor measurement data 160), from which predictive data 164 will be generated (e.g., for performing correction actions based on historical data).

[0045] Component data 152 may include component values ​​and identifiers 158. Sensor data 154 may include sensor values ​​and identifiers 158. Measurement data 156 may include measurement values ​​and identifiers 158. Each instance (e.g., set) of component data 152, sensor data 154, and measurement data 156 may correspond to a corresponding substrate processing device 134 (e.g., associated with a substrate device identifier), a corresponding component of the substrate processing device 134 (e.g., associated with a component identifier), a corresponding product carrier (e.g., associated with an identifier), a corresponding timestamp (e.g., associated with an identifier), and / or a corresponding product (e.g., associated with a product identifier).

[0046] In some embodiments, client device 106 may store current data 162 (e.g., component, sensor, and / or measurement data received after the generation of aggregated component sensor measurement data 160) in data storage 150, and prediction server 112 may retrieve current data 162 from data storage 150. In some embodiments, prediction server 112 may store the output of trained machine learning model 190 (e.g., prediction data 164) in data storage 150, and client device 106 may retrieve the output from data storage 150.

[0047] In some embodiments, the prediction system 110 further includes server machines 170 and 180. Server machine 170 includes a dataset generator 172 capable of generating datasets (e.g., a set of data inputs, a set of data inputs, and a set of target outputs) to train, validate, and / or test the machine learning model 190. Some operations of the dataset generator 172 are described in detail below with reference to Figures 3A and 4A. In some embodiments, the dataset generator 172 may divide aggregated component sensor measurements 160 into a training set (e.g., 60% of the aggregated component sensor measurements 160), a validation set (e.g., 20% of the aggregated component sensor measurements 160), and a test set (e.g., 20% of the aggregated component sensor measurements 160). In some embodiments, the prediction system 110 (e.g., via prediction component 114) generates multiple sets of features. For example, the first feature set may be a first data set (e.g., from a first sensor set) corresponding to each of the data sets (e.g., training set, validation set, and test set), and the second feature set may be a second set of data of various types corresponding to each of the data sets (e.g., from a second sensor set different from the first sensor set).

[0048] Server machine 180 includes a training engine 182, a verification engine 184, a selection engine, and / or a testing engine 186. Engines (e.g., training engine 182, verification engine 184, selection engine 185, and testing engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (such as instructions that run on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be able to train machine learning model 190 using one or more sets of features associated with a training set from dataset generator 172. Training engine 182 may generate multiple trained machine learning models 190, each trained machine learning model 190 corresponding to a dissimilar set of features from the training set (e.g., sensor data from a dissimilar set of sensors). For example, a first trained machine learning model may have been trained using all features (e.g., X1-X5), a second trained machine learning model may have been trained using a first subset of features (e.g., X1, X2, X4), and a third trained machine learning model may have been trained using a second subset of features (e.g., X1, X3, X4, and X5), which may partially overlap with the first subset of features.

[0049] The validation engine 184 may be able to validate the trained machine learning model 190 using a corresponding set of features from the validation set generated by the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features from the training set may be validated using a first set of features from the validation set. The validation engine 184 may determine the accuracy of each trained machine learning model 190 based on the corresponding set of features from the validation set. The validation engine 184 may discard trained machine learning models 190 with accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be able to select one or more trained machine learning models 190 with accuracy that meets a threshold accuracy. In some embodiments, the selection engine 185 may be able to select the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190.

[0050] The testing engine 186 may be able to test the trained machine learning model 190 using a corresponding set of features from the test set generated by the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features from the training set may be tested using a first set of features from the test set. The testing engine 186 may determine, based on the test set, that the trained machine learning model 190 has the highest accuracy among all trained machine learning models.

[0051] Machine learning model 190 may refer to a model artifact created by training engine 182 using a training set that includes data inputs and, in some embodiments, a corresponding target output (the correct answer to the corresponding training input). Patterns in the dataset can be discovered that cluster the data inputs and / or map the data inputs to the target output (the correct answer), and provide the machine learning model 190 with a mapping that captures these patterns. Machine learning model 190 may use one or more of the following: linear regression, random forest, neural networks (e.g., artificial neural networks).

[0052] The prediction component 114 can provide current data 162 to a trained machine learning model 190 and can run the trained machine learning model 190 on the input to obtain one or more outputs. The prediction component 114 may be able to determine (e.g., extract) the prediction data 164 from the output of the trained machine learning model 190 and may be able to determine (e.g., extract) confidence data indicating that the prediction data 164 corresponds to a confidence level of a component and / or substrate of the substrate processing device 134 from the output. The prediction component 114 or the correction action component 108 may use the confidence data to determine, based on the prediction data 164, whether to cause a correction action associated with the substrate processing device 134.

[0053] Confidence data may include or indicate the confidence level of predicted data 164 corresponding to current data 162. In one instance, the confidence level is a real number between 0 and 1 (inclusive), where 0 indicates that there is no confidence level of predicted data 164 corresponding to current data 162, and 1 indicates that the absolute confidence level of predicted data 164 corresponding to current data 162. In some embodiments, system 100 may use prediction system 110 to determine predicted data 164 instead of using measurement device 144 to determine measurement data. In some embodiments, in response to confidence data indicating a confidence level below a threshold level, system 100 may cause measurement device 144 to generate measurement data. In response to confidence data indicating a confidence level below a threshold level for a predetermined number of instances (e.g., percentage of instances, frequency of instances, total number of instances, etc.), prediction component 114 may cause retraining of trained machine learning model 190 (e.g., based on current data 162 and performance data 166 corresponding to current data 162, etc.).

[0054] For illustrative and not limiting purposes, this embodiment describes training a machine learning model using aggregated component sensor measurements 160 and inputting current data 162 into the trained machine learning model to determine prediction data 164. In other embodiments, a heuristic or rule-based model is used to determine prediction data 164 (e.g., without using a trained machine learning model). Prediction component 114 may monitor aggregated component sensor measurements 160. Any of the information described with respect to data input 301 of Figure 3A may be monitored or otherwise used in a heuristic or rule-based model.

[0055] In some embodiments, the functionality of the data integration server 102, client device 106, prediction server 112, sensor server 132, metering server 142, server machine 170, and server machine 180 can be provided by a smaller number of machines. For example, in some embodiments, server machines 170 and 180 may be integrated into a single machine, while in other embodiments, server machine 170, server machine 180, and prediction server 112 may be integrated into a single machine. In some embodiments, sensor server 132, metering server 142, and data integration server 102 may be integrated into a single machine.

[0056] Typically, the functions described in one embodiment as being performed by the data integration server 102, client device 106, prediction server 112, sensor server 132, metering server 142, server machine 170, and server machine 180 can also be performed on the prediction server 112 in other embodiments, where appropriate. Furthermore, the functionality attributed to a particular component can be performed by different or multiple components operating together. For example, in some embodiments, the prediction server 112 may determine a correction action based on prediction data. In another instance, the client device 106 may determine prediction data 164 based on the output from a trained machine learning model.

[0057] Furthermore, the functionality of a particular component can be performed by different or multiple components operating together. One or more of the prediction servers 112, 170, or 180 can be accessed as a service, which is provided to other systems or devices via an appropriate application programming interface (API).

[0058] In this embodiment, "user" may refer to a single individual. However, other embodiments of this invention encompass "user" as an entity controlled by a plurality of users and / or automated sources. For example, a collection of independent users as a group of administrators may be considered a "user".

[0059] Although the embodiments of this case describe generating aggregated component sensor measurement data 160 to perform calibration operations in a manufacturing facility (e.g., a substrate manufacturing facility), the embodiments can also be generally applied to the type of data aggregation to perform operations. The embodiments can generally be applied to integrating different types of data. For example, sensor data can be aggregated with corresponding component failure data to predict the end of component lifespan. In another example, images can be aggregated with corresponding image classifications to predict image classification.

[0060] Figure 2A is a flowchart of a method 200A for integrating components, sensors, and measurement data according to certain embodiments. Method 200A can be executed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (such as instructions that run on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. In one embodiment, method 200A may be executed in part by a data integration server 102 (e.g., data integration component 104). In some embodiments, non-transitory storage media storage instructions, when executed by a processing device (e.g., the processing device of data integration server 102), cause the processing device to execute method 200A.

[0061] For simplicity, method 200A is depicted and described as a series of actions. However, the actions according to this invention can occur in various orders and / or simultaneously, and other actions are not presented or described herein. Furthermore, not all the actions shown can be performed to achieve method 200A according to the disclosed object. In addition, those skilled in the art will understand and recognize that method 200A can alternatively be represented by a state diagram or events as a series of interrelated states.

[0062] Referring to Figure 2A, at block 202, the processing logic receives (e.g., from component server 122, from data storage 150) a collection of component data associated with components of the substrate processing apparatus. Identifiers may be assigned to components. In some embodiments, at least two unique identifiers are assigned to a component, including a component number (e.g., including a revision number) and a serial number. After component manufacturing, dimensions (e.g., size, weight, surface roughness, feature location, feature geometry, hole diameter, hole location, hole roundness, difference between the minimum and maximum holes in the component, range of hole diameters in the component, surface topology, hardness tester, hardness, etc.) may be measured (e.g., via automated optical inspection (AOI), atomic force microscope (AFM), and / or coordinate measurement (CMM)). Component data may include information about the following: where the component is manufactured, how the component is manufactured (e.g., manufacturing process, machining, grinding, joining, welding, cleaning, brazing, etc.), when the component is manufactured, curing time, environmental conditions for storage and transportation of the component (e.g., temperature, humidity, pressure, etc.), how long the component is stored, when the component is installed, measurements of the component, one or more identifiers of the component, etc.

[0063] Component data may include component measurement data and may be stored in a manufacturing database as part numbers and serial numbers. In response to the mounting of a component into a substrate processing apparatus (e.g., a semiconductor manufacturing system), the part number, serial number, and date (e.g., mounting date) may be recorded in a general database. The general database may copy component measurement data from the manufacturing database (e.g., using part numbers and / or serial numbers to identify component measurement data) into the general database.

[0064] In some embodiments, component data includes AOI measurements, such as via configurations (e.g., providing light on the back side and measurement illumination on the other side, which can be used for drilling), blind via configurations (e.g., light on the same side as the measurement, which can be used after brazing), backlighting (e.g., capturing images and fitting circles), front (or ring) lighting, etc. Focus, contrast, and image analysis can be controlled to optimize the results.

[0065] The set of component data may include component manufacturing data (e.g., manufacturing parameters, processing parameters, hardware parameters, etc. of the component manufacturing equipment), component measurement data, and / or component material property data. At least a portion of the set of component data may be measured via one or more AOI equipment, AFM, and / or CMM machines. Each part of the set of component data may include component values ​​and component identifiers associated with the production of the component.

[0066] Component data 152 may include multiple sets of component data, wherein each set of component data includes a component value and one or more identifiers and timestamps (e.g., operating date, operating hour, etc., corresponding to when the component was processed by component manufacturing equipment 124).

[0067] In some embodiments, for each set of component data 152, component server 122 may receive component values ​​from sensor 126 (e.g., associated with component manufacturing operations of component manufacturing equipment 124 that produces the components) and may receive identifiers from identifier reader 128.

[0068] Identifiers can be added to the file of component data 152 as fields or attributes. In some embodiments, component server 122 stores identifier information in the file of component data 152.

[0069] At block 204, the processing logic receives (e.g., from sensor server 123, from data storage 150) a set of sensor data associated with a substrate processing operation performed by the substrate processing device to generate a substrate. Each of the sets of sensor data may include a sensor value and a sensor data identifier associated with generating the substrate by the substrate processing device.

[0070] As the substrate is generated by substrate processing equipment (e.g., the substrate is running on a semiconductor manufacturing system), sensor data (e.g., pressure, temperature, time, gas flow, etc.) along with tracking information on the wafer (e.g., FOUP number, slot number, and date) are recorded in the sensor database.

[0071] The sensor data 154 may include multiple sets of sensor data, wherein each set of sensor data includes sensor values ​​and one or more identifiers (e.g., sensor data identifiers such as LOT_ID, sensor carrier identifiers such as FOUP ID, product identifiers), and timestamps (e.g., operating date, operating hour, etc., corresponding to when the product was processed by the substrate processing device 134).

[0072] In some embodiments, for each set of sensor data 154, sensor server 132 may receive sensor values ​​from sensors 136 (e.g., associated with substrate processing operations of substrate processing apparatus 134 that generates the substrate) and may receive identifiers (e.g., sensor carrier identifiers such as FOUP ID) from identifier readers 138 (e.g., FOUP RFID readers) associated with substrate carriers (e.g., FOUPs) of the substrate. For each set of sensor data, sensor server 132 may generate an identifier (e.g., LOT_ID) based on the sensor carrier identifier and a timestamp (e.g., by concatenating at least a portion of the sensor carrier identifier and at least a portion of the timestamp). For example, for a sensor carrier identifier of FOUP ID 3044 and a timestamp of November 14, 2018, sensor server 132 may generate a sensor data identifier of F3044_111418. The concatenation character can be any character, such as "-", "+", etc., as chosen or convenient for the software architecture.

[0073] Identifiers can be added to the sensor data 154 file as fields or attributes. In some embodiments, the sensor server 132 stores carrier identifier (e.g., FOUP ID) information on the sensor data 154 file. In some embodiments, the sensor server 132 automatically creates a unique sensor data identifier (e.g., LotID) based on the FOUP ID plus a date stamp plus a timestamp.

[0074] At block 206, the processing logic receives (e.g., from metering server 142, from data storage 150) a set of metering data of a substrate generated by a substrate processing apparatus including components, performed through substrate processing operations. Each piece of the set of metering data may include a metering value and a component identifier associated with a component generated by the substrate processing apparatus.

[0075] After the substrate is processed, the substrate is measured on a metrology tool (e.g., metrology equipment) and the substrate measurements (e.g., thickness, film thickness, layer thickness, refractive index, particle count, wafer metrology data, imaging data, etc.) along with wafer tracking information (e.g., FOUP number, slot number, and date) are recorded in a metrology database.

[0076] Measurement data 156 may include multiple sets of measurement data, wherein each set of measurement data includes a measurement value and an identifier. Each identifier may include a measurement carrier identifier (e.g., FOUP ID), a product identifier, and / or a timestamp (e.g., date, hour, etc., corresponding to when the measurement was taken).

[0077] In some embodiments, for each set of measurement data 156, the measurement server 142 may receive measurement values ​​from the measurement device 144 (e.g., associated with a substrate generated by a substrate processing operation of the substrate processing device 134) and may receive identifiers (e.g., measurement carrier identifiers, such as FOUP IDs) from an identifier reader 146 (e.g., a FOUP RFID reader) associated with a product carrier (e.g., FOUP). For each set of measurement data, the measurement server 142 may generate an identifier 158 based on the measurement carrier identifier and a timestamp. For example, for the measurement carrier identifier of F3044 (e.g., which matches the sensor carrier identifier of the corresponding sensor data 154) and the timestamp of November 14, 2018, the sensor server 132 may generate a measurement data identifier of F3044_111418. The measurement data 156 may be stored in a measurement file containing measurement data identifiers (e.g., LOT_ID) and additional contextual information (e.g., product identifiers, etc.).

[0078] In some embodiments, the same product carrier (e.g., FOUP) may be associated with the same product in the sensor system 130 (e.g., performing substrate processing on a substrate) and the metrology system 140 (e.g., measuring a wafer). For the same product, the sensor carrier identifier and the metrology carrier identifier may be the same carrier identifier.

[0079] In some embodiments, the metering server 142 may determine whether the metering data 156 corresponds to a pre-measurement (e.g., metering data 156 provided before substrate processing of the execution substrate during which sensor data 154 is obtained) or a post-measurement (e.g., metering data 156 provided after substrate processing of the execution substrate during which sensor data 154 is obtained). The metering server 142 may include an indication in the identifier 158 indicating whether the metering data 156 corresponds to a pre-measurement or a post-measurement. For example, the file name corresponding to the pre-measurement metering data 156 may contain "PRE", and a file name without "PRE" may be considered a post-measurement.

[0080] In some embodiments, sensor data 154 corresponding to a product set is provided from sensor 136 to sensor server 132 at a first time point, and metering data 156 corresponding to the same product set is provided at a second time point. In some embodiments, the first and second time points correspond to the same date (e.g., the same date stamp). In some embodiments, the first and second time points are the most recent time points for a matching corresponding carrier identifier (e.g., for the same FOUP ID). In some embodiments, for metering data 156 that is to be measured in advance, the second time point is the fastest time point before the first time point for a matching corresponding carrier identifier (e.g., for the same FOUP ID). In some embodiments, for a portion of metering data 156 that is to be measured later, the second time point is the fastest time point after the first time point for a matching corresponding carrier identifier (e.g., for the same FOUP ID).

[0081] In some embodiments, an identifier 158 (e.g., a FOUP ID plus a timestamp) is automatically generated by the component server 122, sensor server 132, and / or metering server 142. In some embodiments, the identifier 158 is generated by user input. For example, the component server 122, sensor server 132, and / or metering server 142 may receive requests (e.g., work sequences) from users (e.g., process engineers) to access (e.g., view, store, etc.) a collection of component data 152, sensor data 154, and / or metering data 156. The component server 122, sensor server 132, and / or metering server 142 may send the collection of metering data 156 to a centralized database for user use. The identifier 158 (e.g., a FOUP ID plus a timestamp, sequence ID) may be a component with a file name created for the component data 152, sensor data 154, and / or metering data 156. The identifier can be created by concatenating information (e.g., FOUP###_DateStamp, such as FOUP ID 3044 running on 11 / 14 / 2018, with the following string in the file name FOUP3044_111418 along with other contextual information). In some embodiments, processing logic (e.g., via data integration component 104) can use a manually entered identifier 158 instead of an automatically generated identifier 158 (e.g., for verifying or spot-checking automatically generated identifiers).

[0082] In some embodiments, at block 208, the processing logic determines a common portion among identifiers (e.g., component identifier, sensor data identifier, metering data identifier) ​​to identify a component-sensor-metering match. In some embodiments, the common portion is a substring (e.g., FOUP3044_111418) between identifiers 158 of component data 152, sensor data 154, and / or metering data 156. In some embodiments, the processing logic searches for and matches the common portion (e.g., substring FOUP3044_111418) on component server 122, sensor server 132, and metering server 142.

[0083] In some embodiments, each common portion (e.g., a substring) includes a matching corresponding carrier identifier (e.g., the same FOUP ID). Each common portion may include the corresponding sensor timestamp (e.g., the operating date) and the corresponding metering timestamp (e.g., the same day, the most recent time, etc.) that is closest to the carrier match.

[0084] In some embodiments, the processing logic identifies a pre-measurement set of measurement data 156 and / or a post-measurement set of measurement data corresponding to the set of sensor data 154. To determine the pre-measurement measurement data (e.g., sensor timestamp) corresponding to the operating date of the set of sensor data 154, the processing logic determines all sets (e.g., measurement files from all measurement devices 144) of measurement data 156 with corresponding timestamps that have a predetermined time frame preceding the operating date (e.g., five days prior to the operating date, a shared portion of the operating date and the preceding five days, etc.). This measurement data matches a carrier identifier (e.g., the same FOUP ID), a product identifier (e.g., the same slot number), and is a pre-measurement (e.g., has "PRE" in the file name). The processing logic may select the measurement data 156 with the corresponding timestamp closest to the operating date as the pre-measurement measurement data to be linked to the sensor data 154.

[0085] To determine the post-measurement data for the operating date, the processing logic identifies all sets (e.g., measurement files from all measurement devices 144) of measurement data 156 with corresponding timestamps for a predetermined time period following the operating date (e.g., five days before the operating date, a shared portion of the operating date and five days after). This measurement data matches the carrier identifier, matches the product identifier, and is a post-measurement (e.g., does not contain "PRE" in the file name). The processing logic may select the measurement data 156 with the closest corresponding timestamp after the operating date as the post-measurement measurement data to be linked to the sensor data 154.

[0086] The processing logic can select one or more sets of measurement data 156 to be linked to the set of sensor data 154 (e.g., most recent pre-measurement measurement data and most recent post-measurement measurement data).

[0087] In some embodiments, sensor data 154 is stored via operations (e.g., substrate processing operations, manufacturing processes). Each set of sensor data 154 may correspond to the same operation. Sensor data 154 may include an identifier corresponding to substrate processing equipment 134 (e.g., tool ID), an identifier corresponding to the manufacturing process (e.g., run ID), a timestamp (e.g., manufacturing process start time, run start time, such as 2018-1114 5:00:00.000), sensor data identifiers (e.g., lot ID, carrier identifier coupled with timestamp, such as F3044_111418), and manufacturing process identifiers (e.g., recipe). An exemplary data structure (e.g., a table) for sensor data 154 is shown in Table 1. Table 1 Tool ID Run ID Run start time LotID plan Tool A Run 1 2018-11-14 5:00:00.000 F3044_111418 Option A Tool A Run 3 2018-11-14 5:10:00.000 F3044_111418 Option A Tool B Run 2 2018-11-14 5:00:00.000 F3044_111418 Option A Tool B Run 4 2018-11-14 5:10:00.000 F3044_111418 Option A Tool B Run 5 2018-11-14 5:20:00.000 F3044_111418 Option A

[0088] In some embodiments, the measurement data 156 is stored in a file that includes a file identifier (e.g., file ID), a file name, and a measurement process (e.g., dielectric measurement, ellipsometer, etc.). The file name may include a carrier identifier (e.g., F3044), a timestamp (e.g., 111418) or a time range (e.g., 111418-11.15.18), and a product identifier (e.g., an indicator slot number, such as "_s1"). An exemplary data structure (e.g., a table) for the measurement data is shown in Table 2. Table 2 File ID File Name Measurement File 1 Data-F3044_111418-11.15.18-Data_s1.csv Ellipsometry File 2 Data-F3044_111418-11.15.18-Data_s2.csv Ellipsometry File 3 Data-F3044_111418-11.15.18-Data_s3.csv Ellipsometry File 4 Data-F3044_111418-11.15.18-Data_s4.csv Ellipsometry File 5 Data-F3044_111418-11.15.18-Data_s5.csv Ellipsometry

[0089] The processing logic can receive sensor data 154 from sensor server 132 and metering data 156 (e.g., metering file data) from metering server 142 (e.g., a shared folder). The processing logic can identify shared attributes (e.g., carrier identifier, timestamp, portion, substring, etc.) between sensor data 154 and metering data 156. Due to slot integrity (e.g., products in the same order in manufacturing and metering processes), the processing logic can identify sensor data 154 corresponding to a product identifier 156 of metering data 154 by the order of sensor data 154 (e.g., a first timestamp of sensor data 154 corresponds to a first product identifier). The processing logic can provide a link between metering data 156 (e.g., metering file) and sensor data 154 (e.g., operating data) by associating file identifiers, manufacturing equipment identifiers, and manufacturing process identifiers. The processing logic can provide a GUI to display link information. Table 3 shows an exemplary data structure (e.g., a table) for the link between sensor data 154 and measurement data 156. Table 3 File ID Tool ID Run ID File 2 Tool A Run 1 File 4 Tool A Run 3 File 1 Tool B Run 2 File 3 Tool B Run 4 File 5 Tool B Run 5

[0090] Table 3 shows that products in slot 2 are processed by tool A at 5:00, products in slot 4 are processed by tool A at 5:10, products in slot 1 are processed by tool B at 5:00, products in slot 3 are processed by tool B at 5:10, and products in slot 5 are processed by tool B at 5:20. The processing logic can generate aggregated component sensor measurement data 160, which includes sensor data 154, measurement data 156, and indications of common attributes (e.g., similar to or different from the link attributes in Table 3).

[0091] In block 210, the processing logic generates a set of aggregated component sensor measurement data (e.g., an aggregated component sensor measurement data structure or table), which includes a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of measurement data. The logic can generate an aggregated set of component sensor measurement data for each component sensor measurement match.

[0092] In some embodiments, sensor data associated with a substrate generated from a sensor database and substrate metrology data from a metrology database are linked by matching tracking information (e.g., FOUP number, slot number, and date). Sensor metrology data can be linked with component data from a manufacturing database by usage dates (e.g., component installation date, component manufacturing date, and substrate processing operation date). The linked data can be output in a format (e.g., aggregated component sensor metrology data) that can be used for manual data processing (e.g., manual calculation, manual processing, using spreadsheets or data analysis software) or automated data processing (e.g., automated calculation, automated processing, automated machine learning models).

[0093] Each set of aggregated component sensor metrology data may include a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of metrology data, each corresponding to a common identifier 158 (e.g., integrated, aggregated, linked, etc.). For example, the aggregated component sensor metrology data structure may include component data, sensor data (e.g., sensor_1 to sensor_N), product identifier (e.g., wafer_ID), carrier identifier (e.g., FOUP_ID), sensor timestamp (e.g., date stamp), metrology data (e.g., thickness, refractive index (RI), dielectric constant (k) etc.), and / or metrology timestamp (e.g., date stamp), wherein the product identifier 148 matches the product identifier, the sensor carrier identifier matches the metrology carrier identifier, and the sensor timestamp matches the metrology timestamp.

[0094] In some embodiments, processing logical joins or links files and product identifiers (e.g., wafer_ID) as shared fields or attributes.

[0095] In block 212, the processing logic causes the collection of aggregated component sensor measurement data to be analyzed to generate an output for performing calibration actions associated with the components of the substrate processing equipment.

[0096] Analyzing the aggregated set of component sensor measurements may include providing the aggregated set of component sensor measurements to train a machine learning model (e.g., see Figure 4B).

[0097] Analyzing the aggregated set of component sensor measurements may include feeding the aggregated set of component sensor measurements to a trained machine learning model and receiving one or more outputs from the trained machine learning model (e.g., see Figure 4C).

[0098] Analyzing a collection of aggregated component sensor metrology data may include performing data mining, determining correlations between data points (e.g., wafer-scale and AOI-scale), root cause amplification, clustering, etc. (e.g., to perform root cause analysis). In some embodiments, analyzing the aggregated collection of component sensor metrology data identifies general areas of problem (e.g., poor uniformity) for further analysis (e.g., determining how to improve uniformity). In some embodiments, analyzing the aggregated collection of component sensor metrology data includes one or more of the following stages: clustering, anomaly detection, correlation analysis, etc. For example, the analysis may be used to determine which component is causing the problem after determining the characteristics of the component causing the problem and after determining the manufacturing parameters causing the problem.

[0099] Analyzing an aggregated set of component sensor measurements may include storing the aggregated set of component sensor measurements (e.g., for training a machine learning model to provide a trained machine learning model capable of generating outputs for performing correction actions). In some embodiments, processing logic stores the aggregated set of component sensor measurements in a data store, a database repository, a technical data repository, etc. Each of the aggregated set of component sensor measurements may be stored in a corresponding data structure (e.g., in a corresponding table, in a corresponding file, etc.).

[0100] In some embodiments, the aggregated component sensor metrology data is stored in a database that is searched based on component data, sensor data, and / or metrology data to determine the optimal component design, component manufacturing parameters, component feature layout, component size, component material properties, etc.

[0101] In some embodiments, aggregated component sensor measurements (e.g., unlabeled data) are stored to train a machine learning model (e.g., unsupervised machine learning, clustering, etc.) to generate a trained machine learning model. The trained machine learning model may be able to produce one or more outputs (e.g., predicted data, indications of where new clustered data belongs).

[0102] In some embodiments, aggregated component sensor measurement data (e.g., labeled data) is stored to train machine learning model 190 (e.g., supervised machine learning) to generate a trained machine learning model. The trained machine learning model may be able to generate one or more outputs (e.g., artificial intelligence (AI) applications, such as generating prediction data 164 based on the input of current data 162) for performing correction actions associated with substrate processing device 134.

[0103] In some embodiments, aggregated component sensor measurement data is stored as cataloged data in a centralized location (e.g., data storage 150) for future use. Conventionally, component data, sensor data, and measurement data are stored in different locations (e.g., different facilities, on a user's laptop, etc.). Hundreds of sensors may exist (e.g., more than 500), thus the component and sensor data that change over time consumes a significant amount of storage capacity (e.g., on a user's laptop) and may eventually be discarded. Method 200A can be used to generate and store aggregated component sensor measurement data 160 over a long period (e.g., months, years) for later analysis to perform calibration actions.

[0104] In some embodiments, the processing logic generates an aggregated set of component sensor measurement data without manual association of component data, sensor data, and measurement data with user input. In some embodiments, the processing logic generates an aggregated set of component sensor measurement data without using an optical character recognition (OCR) system.

[0105] In some embodiments, each product carrier has multiple time slots carrying products (e.g., substrates), and time slot integrity (e.g., the order of products in the time slots) can be maintained when the products are processed by substrate processing equipment 134 (e.g., on a chamber), and time slot integrity is transmitted for metrological measurement by metrological equipment 144 (e.g., the first product to be processed by substrate processing equipment 134 is the first product to be measured by metrological equipment 144).

[0106] In some embodiments, corrective actions include updating the design of a component (e.g., eliminating component manufacturing operations), the quality of the component (e.g., reducing component variations, such as improving the diameter, location, and / or roundness of holes), the dimensions of the component (e.g., updating the thickness and / or diameter of a portion of the component), the feature layout of the component (e.g., updating the location of holes on the component), and the component manufacturing operations used to produce the component (e.g., replacing or avoiding certain manufacturing operations). In some embodiments, corrective actions include performing root cause analysis to determine updates in the component or updates to the substrate processing operations. In some embodiments, corrective actions include causing a GUI alarm, interrupting the operation of the component manufacturing equipment or the substrate processing equipment (e.g., stopping the substrate processing equipment 134, stopping a portion of the substrate processing equipment 134, not performing the manufacturing process, etc.), and / or causing updates to the manufacturing parameters of the component manufacturing equipment or the substrate processing equipment. In some embodiments, corrective actions include updating the substrate processing operations (e.g., adjusting operating conditions to obtain substrate output).

[0107] In some embodiments, method 200A is used to determine differences (e.g., component differences, manufacturing parameter differences, etc.) between similar substrate processing devices that result in different performance.

[0108] Due to the calibration action, the new component may have better quality (e.g., reduced variation, reduced hole size variation, etc.) and / or the substrate may have better quality (e.g., measurement within the threshold).

[0109] Figure 2B is a block diagram 200B integrating components, sensors, and measurement data according to certain embodiments. Parts of block diagram 200B may be similar to or the same as the blocks of method 200A in Figure 2A.

[0110] Block diagram 200B may include a component server 222, a component system 220, a sensor server 232 (e.g., an FSS, sensor server 132 in Figure 1), a sensor system 230 (e.g., a tooling / processing system, sensor system 130 in Figure 1), a metering server 242, and a metering system 240 (e.g., metering system 140 in Figure 1). The component system 220 may include a component manufacturing apparatus 224, a sensor 226 (e.g., sensor 126 in Figure 1), and an identifier reader 228. The sensor system 230 may include a substrate processing apparatus 234 (e.g., substrate processing apparatus 134 in Figure 1), a sensor 236 (e.g., sensor 136 in Figure 1), and an identifier reader 238 (e.g., a FOUP RFID reader, identifier reader 138 in Figure 1). The metering system 240 may include a metering device 244 (e.g., the metering device 144 in Figure 1) and an identifier reader 246 (e.g., a FOUP RFID reader, the identifier reader 146 in Figure 1).

[0111] Component server 222 can receive component values ​​and component identifiers from component system 220 to generate component data 252 (component data 152 in Figure 1). Sensor server 232 can receive sensor values ​​and sensor data identifiers from sensor system 230 to generate sensor data 254 (sensor data 154 in Figure 1). Metering server 242 can receive metering values ​​and metering data identifiers from metering system 240 to generate sensor data 254 (sensor data 154 in Figure 1).

[0112] Data integration server 102 (e.g., via data integration component 104) may receive component data 252 (e.g., see block 202), sensor data 254 (e.g., see block 204), and metering data 256 (e.g., see block 206). In some embodiments, data integration server 102 may receive user input (e.g., manual input) of FOUP ID and DateStamp from user data storage. User input may include manual login of an identifier (e.g., metering data identifier). Data integration server 102 may compare metering data received from metering server 242 with user input to verify the accuracy (e.g., spot check) of metering data 256.

[0113] Data integration server 102 can match common parts (e.g., substring FOUP####_DateStamp) (e.g., see block 208). Data integration server 102 can generate a collection of aggregated component sensor measurement data based on the common parts (e.g., see block 210) (e.g., files in linked / joined data structures such as tables). Data integration server 102 can store aggregated component sensor measurement data 260 (e.g., aggregated component sensor measurement data 160 in Figure 1) in a data storage (e.g., data storage 150 in Figure 1, technical data warehouse (TDW), etc.).

[0114] Figure 3A illustrates an exemplary data set generator 372 (e.g., data set generator 172 of Figure 1) for generating a machine learning model (e.g., model 190 of Figure 1) using aggregated component sensor measurement data 360 (e.g., aggregated component sensor measurement data 360 of Figure 1) according to certain embodiments. System 300A of Figure 3A illustrates data set generator 372, data input 301, and target output 303.

[0115] In some embodiments, dataset generator 372 generates datasets (e.g., training sets, validation sets, test sets) that include one or more data inputs 301 (e.g., training inputs, validation inputs, test inputs). In some embodiments, the dataset further includes one or more target outputs 303 corresponding to the data inputs 301. The dataset may also include mapping data that maps the data inputs 301 to the target outputs 303. The data inputs 301 may also be referred to as "features," "attributes," or "information." In some embodiments, dataset generator 372 may provide the dataset to training engine 182, validation engine 184, or testing engine 186, wherein the dataset is used to train, validate, or test machine learning model 190. Some embodiments of generating training sets may be further described with reference to Figure 4A.

[0116] In some embodiments, the dataset generator 372 generates a data input 301 based on aggregated component sensor metering data 360. In some embodiments, the dataset generator 372 generates a target output 303 based on performance data 366 associated with the aggregated component sensor metering data 360. The dataset generator 372 may determine the mapping from each set of aggregated component sensor metering data 360 to the performance data 366.

[0117] In some embodiments, data input 301 may include one or more sets of features of aggregated component sensor measurement data 360. Each instance of aggregated component sensor measurement data 360 may include component data for one or more types of components, sensor data from one or more types of sensors, and / or measurement data associated with one or more types of substrates.

[0118] In some embodiments, the data set generator 272 may generate a first data input corresponding to a first set of features to train, validate, or test a first machine learning model, and the data set generator 272 may generate a second data input corresponding to a second set of features to train, validate, or test a second machine learning model.

[0119] In some embodiments, dataset generator 272 may discretize one or more of the data input 301 or the target output 303 (e.g., in a classification algorithm for a regression problem). Discretization of the data input 301 or the target output 303 can transform continuous values ​​of variables into discrete values. In some embodiments, the discrete values ​​of the data input 301 indicate discrete manufacturing parameters to obtain the target output 303 (e.g., discrete-property data).

[0120] The data input 301 and target output 303 used to train, validate, or test machine learning models may include information from a specific facility (e.g., a specific component manufacturing facility or substrate manufacturing facility). For example, aggregated component sensor measurement data 360 and performance data 366 may be used in the same manufacturing facility.

[0121] In some embodiments, information for training the machine learning model may come from specific types of manufacturing equipment (e.g., component manufacturing equipment 124, substrate processing equipment 134) of a manufacturing facility with specific characteristics, and the trained machine learning model may determine the results of a particular group of manufacturing equipment based on inputs of aggregated component sensor measurement data 360 associated with shared characteristics of one or more components in a particular group. In some embodiments, information for training the machine learning model may be available from components from two or more manufacturing facilities, and the trained machine learning model may determine the results of a component based on inputs from one manufacturing facility.

[0122] In some embodiments, after generating a dataset and using the dataset to train, validate, or test the machine learning model 190, the machine learning model 190 may be further trained, validated, or tested (e.g., with additional aggregated component sensor measurement data 160 and performance data 166 of Figure 1) or adjusted (e.g., adjusting weights associated with the input data of the machine learning model 190, such as connection weights in a neural network).

[0123] Figure 3B is a block diagram illustrating a system 300 for generating prediction data 364 (e.g., prediction data 164 in Figure 1) according to certain embodiments. System 300B can be used to determine correction actions associated with components of the substrate processing device 134 based on aggregated component sensor measurement data 360 (e.g., aggregated component sensor measurement data 360 in Figure 1).

[0124] At block 310, system 300 (e.g., prediction system 110 of Figure 1) performs data partitioning of aggregated component sensor measurement data 360 (e.g., aggregated component sensor measurement data 160 of Figure 1) (and in some embodiments, performance data 366) (e.g., via dataset generator 172 of server machine 170 of Figure 1) to generate training set 302, validation set 304, and test set 306. For example, the training set may be 60% of the aggregated component sensor measurement data 360, the validation set may be 20% of the aggregated component sensor measurement data 360, and the test set may be 20% of the aggregated component sensor measurement data 360. System 300 may generate multiple sets of features for each of the training set, validation set, and test set. For example, if the aggregated component sensor measurement data 360 is associated with 20 sensors (e.g., sensor 126 in Figure 1) and 100 products (e.g., wafers each corresponding to sensor data from the 20 sensors), the first set of features could be sensors 1-10, the second set of features could be sensors 11-20, the training set could be products 1-60, the validation set could be products 61-80, and the test set could be products 81-100. In this example, the first set of features for the training set would be from sensors 1-10 of products 1-60.

[0125] At block 312, system 300 performs model training using training set 302 (e.g., via training engine 182 in Figure 1). System 300 can use multiple sets of features from training set 302 (e.g., a first set of features from training set 302, a second set of features from training set 302, etc.) to train multiple models. For example, system 300 can use a first set of features from the training set (e.g., sensor data from sensors 1-10 of products 1-60) to train a machine learning model to produce a first trained machine learning model, and use a second set of features from the training set (e.g., sensor data from sensors 11-20 of products 1-60) to produce a second trained machine learning model. In some embodiments, the first trained machine learning model and the second trained machine learning model can be combined to produce a third trained machine learning model (e.g., which may be a predictor that is better than the first or second trained machine learning model itself). In some embodiments, the feature sets used in the comparison models can overlap (e.g., the first set of features comes from sensors 1-15 and the second set of features comes from sensors 5-20). In some embodiments, hundreds of models may be generated, including models with various permutations of features and combinations of models.

[0126] At block 314, system 300 performs model validation using validation set 304 (e.g., via validation engine 184 in Figure 1). System 300 can use a corresponding set of features from validation set 304 to validate each of the trained models. For example, system 300 can use a first set of features in the validation set (e.g., sensor data from sensors 1-10 of products 61-80) to validate a first trained machine learning model, and a second set of features in the validation set (e.g., sensor data from sensors 11-20 of products 61-80) to validate a second trained machine learning model. In some embodiments, system 300 can validate hundreds of models generated at block 312 (e.g., models with various permutations of features, combinations of models, etc.). At block 314, system 300 can determine the accuracy of each of one or more trained models (e.g., via model validation) and can determine whether one or more trained models have an accuracy that meets a threshold accuracy. In response to the determination that none of the trained models have the accuracy required to meet the threshold, the process returns to block 312, where system 300 performs model training using different sets of features from the training set. In response to the determination that one or more trained models have the accuracy required to meet the threshold, the process continues to block 316. System 300 may discard trained machine learning models with accuracy below the threshold (e.g., based on the validation set).

[0127] At block 316, system 300 performs model selection (e.g., via selection engine 185 in Figure 1) to determine which of one or more trained models that meets the threshold accuracy has the highest accuracy (e.g., selected model 308, based on validation at block 314). In response to the determination that two or more trained models meeting the threshold accuracy have the same accuracy, the process may return to block 312, where system 300 performs model training using a further refined training set corresponding to a further refined feature set to determine the trained model with the highest accuracy.

[0128] At block 318, system 300 performs model testing (e.g., via test engine 186 of Figure 1) using test set 306 to test the selected model 308. System 300 can use a first set of features from the test set (e.g., sensor data from sensors 1-10 of products 81-100) to test the first trained machine learning model to determine if the first trained machine learning model meets a threshold accuracy (e.g., based on the first set of features from test set 306). In response to the selected model 308's accuracy not meeting the threshold accuracy (e.g., the selected model 308 overfits to training set 302 and / or validation set 304 and is not applicable to other datasets, such as test set 306), the process continues to block 312, where system 300 performs model training (e.g., retraining) using different training sets corresponding to different sets of features (e.g., sensor data from different sensors). In response to determining that the selected model 308 has accuracy meeting the threshold accuracy based on test set 306, the process continues to block 320. In at least block 312, the model can learn patterns in the aggregated component sensor measurement data 360 to make predictions, and in block 318, the system 300 can apply the model to the remaining data (e.g., test set 306) to test the predictions.

[0129] At block 320, system 300 uses a trained model (e.g., selected model 308) to receive current data 362 (e.g., current data 162 of Figure 1, current aggregated component sensor measurement data 160 of Figure 1) and determines (e.g., extracts) prediction data 364 (e.g., prediction data 164 of Figure 1) from the output of the trained model to perform correction actions associated with the components of component manufacturing equipment 124.

[0130] In some embodiments, performance data 366 corresponding to current data 362 is received and model 308 is retrained based on current data 362 and performance data 366.

[0131] In some embodiments, one or more operations of blocks 310-320 may occur in various orders and / or other operations not presented or described herein. In some embodiments, one or more operations of blocks 310-320 may not be performed. For example, in some embodiments, one or more of the data partitioning of block 310, the model verification of block 314, the model selection of block 316, or the model testing of block 318 may not be performed.

[0132] Figures 4A through 4C are flowcharts of methods 400A-C associated with the integration of components, sensors, and measurement data according to certain embodiments. Methods 400A-C can be executed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (such as instructions running on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, method 400A may be partially executed by prediction system 110 (e.g., server engine 170, dataset generator 172, etc.). According to embodiments of this invention, prediction system 110 may use method 400A to train, validate, or test at least one of a machine learning model. In some embodiments, as described with respect to Figures 1 and 3A, one or more operations of method 400A may be executed by dataset generator 172 of server machine 170. In some embodiments, methods 400B-C may be performed in part by prediction system 110 (e.g., prediction server 112, prediction component 114, etc.). According to embodiments of the present invention, prediction system 110 may use method 400B to train a machine learning model. According to embodiments of the present invention, prediction system 110 may use method 400C to utilize the trained machine learning model. In some embodiments, as described with respect to Figures 1 and 3B, one or more operations of methods 400B-C may be performed by prediction component 114 of prediction server 112. Note that components described with respect to Figures 1, 2, or 3A through 3B may be used to illustrate the state of Figures 4A through 4C. In some embodiments, non-transitory storage media storage instructions, when executed by a processing device (e.g., prediction system 110), cause the processing device to perform methods 400A-C.

[0133] For simplicity, methods 400A-C are depicted and described as a series of actions. However, the actions according to this invention can occur in various orders and / or simultaneously, and other actions are not presented or described herein. Furthermore, not all the actions shown can be performed to achieve methods 400A-C according to the disclosed object. In addition, those skilled in the art will understand and recognize that methods 400A-C can alternatively be represented by a state diagram or events as a series of interrelated states.

[0134] Referring to Figure 4A, method 400A is associated with a dataset for generating a machine learning model for performing correction actions associated with components of the substrate processing equipment.

[0135] In block 402, the processing logic of method 400A initializes the training set T to an empty set.

[0136] At block 404, the processing logic generates a first data input (e.g., a first training input, a first verification input), which includes aggregated component sensor measurement data (e.g., aggregated component sensor measurement data 160 in Figure 1). In some embodiments, the first data input may include a first set of features of various types of aggregated component sensor measurement data, and the second data input may include a second set of features of various types of aggregated component sensor measurement data (e.g., as described with respect to Figure 3A). The processing logic may generate data input based on portions of the aggregated component sensor measurement data 160 (e.g., extracting component data 152, sensor data 154, and / or measurement data 156 from the data structure or table of the aggregated component sensor measurement data 160).

[0137] In some embodiments, at block 406, the processing logic generates a first target output for one or more data inputs (e.g., a first data input). The first target output may be corresponding performance data (e.g., performance data 366, a label on a substrate associated with aggregated component sensor metering data 160). The processing logic may generate the target output based on the aggregated component sensor metering data 160 (e.g., extracting component data 152, sensor data 154, and / or metering data 156 from the data structure or table of the same aggregated component sensor metering data 160 from which the data inputs are extracted).

[0138] At block 408, the processing logic generates, as appropriate, mapping data indicating the input / output mapping. The input / output mapping (or mapping data) may refer to data inputs (e.g., one or more data inputs as described herein), the target output of the data inputs (e.g., where the target output identifies performance data 166), and the association between the data inputs and the target output. The mapping data may be a shared data structure or table based on aggregated component sensor measurement data 160 from which the data inputs and target outputs are extracted.

[0139] At block 410, the processing logic will add the data input generated at block 404 and / or the mapping data generated at block 408 to the data set T.

[0140] At block 412, the processing logic branches based on whether the dataset T is sufficient for at least one of training, validating, and / or testing the machine learning model 190. If so, execution proceeds to block 414; otherwise, execution returns to block 404. In some embodiments, the sufficiency of the dataset T may be determined simply based on the number of input / output mappings in the dataset, while in some other embodiments, in addition to or instead of the number of input / output mappings, the sufficiency of the dataset T may be determined based on one or more other criteria (e.g., a measure of the diversity of data instances, accuracy, etc.).

[0141] At block 414, the processing logic provides a dataset T (e.g., to server machine 180) to train, validate, and / or test the machine learning model 190. In some embodiments, the dataset T is a training set and is provided to a training engine 182 on server machine 180 to perform training. In some embodiments, the dataset T is a validation set and is provided to a validation engine 184 on server machine 180 to perform validation. In some embodiments, the dataset T is a test set and is provided to a test engine 186 on server machine 180 to perform testing. In the case of a neural network, for example, input values ​​(e.g., numerical values ​​associated with data input 301) for a given input / output mapping are input to the neural network, and output values ​​(e.g., numerical values ​​associated with target output 303) for the input / output mapping are stored in the output nodes of the neural network. The connection weights in the neural network are then adjusted according to a learning algorithm (e.g., backpropagation, etc.), and the procedure is repeated for other input / output mappings in the dataset T. Following block 414, a machine learning model (e.g., machine learning model 190) may be trained using training engine 182 of server machine 180, validated using verification engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained machine learning model may be implemented by prediction component 114 (of prediction server 112) to generate prediction data 164 for performing correction actions associated with components of substrate processing equipment 134.

[0142] Referring to Figure 4B, method 400B is associated with a trained machine learning model for performing correction actions associated with components of the substrate processing apparatus.

[0143] In block 420, the processing logic identifies aggregated component sensor measurement data associated with components of the substrate processing equipment. The aggregated component sensor measurement data can be generated from component data, sensor data from substrate processing operations performed by the substrate processing equipment including the components to produce the substrate, and substrate measurement data. The set of aggregated component sensor measurement data can be historical data corresponding to already manufactured components and already produced substrates.

[0144] In some embodiments, at block 422, processing logic identifies performance data corresponding to aggregated component sensor measurement data. In some embodiments, the performance data indicates whether measurements of the substrate associated with the aggregated component sensor measurement data (e.g., dimensions, images, etc.) meet threshold measurements (e.g., whether they are good or bad).

[0145] In block 424, the processing logic uses data inputs including aggregated component sensor measurement data (e.g., and target output including performance data) to train a machine learning model to produce a trained machine learning model, which is configured to produce outputs that cause the execution of correction actions associated with components of the substrate processing equipment.

[0146] In some embodiments, the machine learning model is trained on data input (e.g., without a target output) to produce an unsupervised trained machine learning model (e.g., with clustered data). In some embodiments, the machine learning model is trained on data input and a target output to produce a supervised trained machine learning model.

[0147] Referring to Figure 4C, method 400C is associated with a machine learning model that uses a component of the substrate processing apparatus to perform correction actions.

[0148] In block 440, the processing logic receives current data. In some embodiments, the current data is the aggregated component sensor measurement data of the current component and / or the current substrate (e.g., performance data is not applicable). In some embodiments, the current data includes component data, sensor data, and / or measurement data.

[0149] In block 442, the processing logic provides current data (e.g., aggregated component sensor measurement data) to a trained machine learning model. The trained machine learning model can be trained using method 400B.

[0150] At block 444, the processing logic obtains one or more outputs indicating predicted data from a trained machine learning model. In some embodiments, the predicted data is predicted performance data (e.g., whether the substrate is good or bad, whether the component is good or bad) (e.g., results from a supervised machine learning model). In some embodiments, the predicted data is an indication of the similarity between historical data and current data (e.g., historical data and current data are part of the same cluster, and current data is not clustered with any historical data) (e.g., results from an unsupervised machine learning model).

[0151] In block 446, the processing logic, based on the predicted data, causes the execution of a correction action associated with a component of the substrate processing apparatus. In some embodiments, the correction action includes adjusting component manufacturing (e.g., design, size, feature layout, manufacturing operations) or substrate production (e.g., substrate processing operations, hardware of the substrate processing apparatus, manufacturing parameters of the substrate processing operations, etc.).

[0152] Figure 5 is a block diagram illustrating a computer system 500 according to certain embodiments. In some embodiments, the computer system 500 may be connected (e.g., via a network, such as a Local Area Network (LAN), intranet, extranet, or the Internet) to other computer systems. The computer system 500 may operate in a client-server network environment within the capacity of a server or client computer, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 500 may be provided by a PC, tablet PC, personal digital assistant (PDA), cellular phone, web device, server, network router, switch or bridge, or any machine capable of executing a set of instructions (continuously or otherwise) specifying actions to be taken by that device. Furthermore, the term "computer" should include any set of computers that independently or jointly execute a set of instructions (or multiple sets of instructions) to perform any one or more methods described herein.

[0153] In another embodiment, the computer system 500 may include a processing device 502 that can communicate with each other via a bus 508, volatile memory 504 (e.g., random access memory (RAM)), non-volatile memory 506 (e.g., read-only memory (ROM) or electrically-erasable programmable ROM (EEPROM)), and a data storage device 516.

[0154] The processing device 502 may be provided by one or more processors, such as general-purpose processors (e.g., complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, very long instruction word (VLIW) microprocessors, microprocessors that implement other types of instruction sets, or microprocessors that implement combinations of various types of instruction sets) or special-purpose processors (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), or network processors).

[0155] The computer system 500 may further include a network interface device 522. The computer system 500 may also include a video display unit 510 (e.g., a liquid crystal display; LCD), an alphanumeric input device 512 (e.g., a keyboard), a cursor control device 514 (e.g., a mouse), and a signal generating device 520.

[0156] In some embodiments, the data storage device 516 may include a non-transitory computer-readable storage medium 524 on which one or more instructions 526 encoding the methods or functions described herein may be stored, including the instruction encoding components of Figure 1 (e.g., data integration component 104, correction action component 108, prediction component 114, etc.) and used to implement the methods described herein.

[0157] Instruction 526 may also be placed entirely or partially within volatile memory 504 and / or processing device 502 by computer system 500 during its execution. Therefore, volatile memory 504 and processing device 502 may also constitute machine-readable storage media.

[0158] Although computer-readable storage media 524 is illustrated as a single medium in the illustrative examples, the term "computer-readable storage media" should include single or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) storing one or more sets of executable instructions. The term "computer-readable storage media" should also include any tangible medium capable of storing or encoding a set of instructions for execution by a computer, which causes the computer to perform any or more of the methods described herein. The term "computer-readable storage media" should include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0159] The methods, components, and features described herein can be implemented by discrete hardware components or integrated into the functionality of other hardware components (such as ASICs, FPGAs, DSPs, or similar devices). Furthermore, the methods, components, and features can be implemented by functional circuitry within firmware modules or hardware devices. Additionally, the methods, components, and features can be implemented in any combination of hardware devices and computer program components, or implemented as a computer program.

[0160] Unless otherwise specifically stated, terms such as “receive,” “generate,” “cause,” “determine,” “update,” “execute,” “store,” “train,” “interrupt,” “select,” “provide,” “display,” or similar, refer to the actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities in the registers and memory of a computer system into other data similarly represented as physical quantities in the memory or registers or other such information storage, transmission, or display devices of a computer system. Furthermore, as used herein, the terms “first,” “second,” “third,” “fourth,” etc., are designations for distinguishing different elements and their numerical designations may not have ordinal meaning.

[0161] The examples described herein also relate to an apparatus for performing the methods described herein. This apparatus may be specially configured to perform the methods described herein, or it may comprise a general-purpose computer system designed by selective programming of a computer program stored in a computer system. Such a computer program may be stored in a computer-readable physical storage medium.

[0162] The methods and illustrative examples described herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized devices to perform the methods described herein and / or each of their individual functions, routines, subroutines, or operations. Examples of structures for various such systems are illustrated in the foregoing description.

[0163] The above description is intended to be illustrative and not restrictive. Although specific illustrative examples and embodiments have been described with reference to this case, it will be appreciated that this case is not limited to the described examples and embodiments. The scope of this case should be determined by referring to the entire scope of the appended claims and their equivalents.

[0164] 100: System 102: Data Integration Server 104: Data Integration Component 106: Client device 108: Correction Motion Component 110: Prediction System 112: Prediction Server 114: Prediction Component 116: Internet 120: Component System 122: Component Server 124: Component manufacturing equipment 126: Sensor 128: Identifier Reader 130: Sensor System 132: Sensor Server 134: Substrate processing equipment 136: Sensor 138: Identifier Reader 140: Metering System 142: Metering Server 144: Measuring equipment 146: Identifier Reader 150: Data storage 152: Component Information 154: Sensor Data 156: Measurement data 158: Identifier 160: Aggregated component sensor measurement data 162: Current Data 164: Forecast Data 166: Performance Data 170: Server Machine 172: Data Set Generator 180: Server Machine 182: Training Engine 184: Verification Engine 185: Select Engine 186: Test Engine 190: Machine Learning Model 200A: Method 200B: Block Diagram 202: Square 204: Square 206: Square 208: Square 210: Square 212: Square 220: Component System 222: Component Server 224: Component manufacturing equipment 226: Sensor 228: Identifier Reader 230: Sensor System 232: Sensor Server 234: Substrate processing equipment 236: Sensor 238: Identifier Reader 240: Metering System 242: Metering Server 244: Measuring equipment 246: Identifier Reader 252: Component Information 254: Sensor Data 256: Measurement data 260: Aggregated component sensor measurement data 300A: System 300B: System 301: Data Input 302: Training Set 303: Target Output 304: Validation set 306: Test Set 308: Selected Model 310: Square 312: Square 314: Square 316: Square 318: Square 320: Square 360: Aggregated component sensor measurement data 362: Current Data 364: Prediction Data 366: Performance Data 372: Data Set Generator 400A:Method 400B: Method 400C: Method 402: Square 404: Square 406: Square 408: Square 410: Square 412: Square 414: Square 420: Square 422: Square 424: Square 440: Square 442: Square 444: Square 446: Square 500: Computer System 502: Processing device 504: Volatile memory 506: Non-volatile memory 508: Busbar 510: Video display unit 512: Alphanumeric Input Device 514: Vernier control device 520: Signal generating device 522: Network Interface Device 524: Non-transitory computer-readable storage media 526: Instruction

[0165] Domestic storage information (please note in order of storage institution, date, and number) none Overseas storage information (please note in the order of storage country, institution, date, and number) none

Claims

1. A method for performing data integration by a processor, comprising the steps of: receiving a plurality of sets of component data associated with a substrate processing device, wherein each of the plurality of sets of component data includes a corresponding component value and a corresponding component identifier, and wherein each of the plurality of sets of component data is associated with hardware parameters of a corresponding device component of the substrate processing device; receiving a plurality of sets of sensor data, wherein each of the plurality of sets of sensor data includes a corresponding sensor value and a corresponding sensor data identifier associated with generating one or more corresponding substrates by the substrate processing device; The method includes receiving a plurality of sets of metrological data, each of which includes a corresponding metrological value associated with one or more corresponding substrates and a corresponding metrological data identifier; generating a plurality of aggregated sets of component sensor metrological data, each of which includes a corresponding set of device component data, a corresponding set of sensor data, and a corresponding set of metrological data; and causing a calibration action associated with the substrate processing device to be performed based on the aggregated sets of component sensor metrological data, wherein the step of causing the calibration action includes the step of using the aggregated sets of component sensor metrological data to train a machine learning model.

2. The method as described in claim 1, wherein the plurality of sets of component data includes one or more of component manufacturing data, component measurement data, or component material property data.

3. The method as described in claim 1, wherein at least a portion of the plurality of sets of component data is measured by one or more of the following: an automated optical inspection (AOI) device, an atomic force microscope (AFM), or a coordinate measuring machine (CMM).

4. The method as described in claim 1, wherein the plurality of sets of sensor data includes one or more of pressure data, temperature data, time data, or gas flow data.

5. The method as described in claim 1, wherein the plurality of sets of metrological data includes one or more of on-wafer metrological data, imaging data, or thickness data.

6. The method as described in Request 1, wherein: Each of the plurality of sets of sensor data is associated with one or more corresponding substrate processing operations performed to generate the one or more corresponding substrates; and each of the plurality of sets of measurement data is associated with the one or more corresponding substrate processing operations performed by the substrate processing apparatus including the corresponding device components.

7. The method as described in claim 6, wherein the step of generating the plurality of sets of aggregated component sensor metering data comprises the following steps: determining a common portion between each corresponding component identifier, each corresponding sensor data identifier, and each corresponding metering data identifier to identify a component sensor metering match; and for each of the component sensor metering matches, generating a corresponding set of aggregated component sensor metering data, the corresponding set comprising a corresponding set of component data corresponding to the corresponding component identifier, a corresponding set of sensor data corresponding to the corresponding sensor data identifier, and a corresponding set of metering data corresponding to the corresponding metering data identifier to generate the plurality of sets of aggregated component sensor metering data.

8. The method as described in claim 1, wherein the step of causing the correction action to be performed comprises the steps of: providing the plurality of sets of aggregated component sensor measurement data to a trained machine learning model and receiving one or more outputs from the trained machine learning model to perform the correction action.

9. The method as claimed in claim 1, wherein the step of causing the execution of the correction action comprises the steps of: storing the plurality of aggregated component sensor measurement data to train a machine learning model to provide a trained machine learning model, and wherein the trained machine learning model is capable of generating one or more outputs to perform the correction action.

10. The method as described in claim 6, wherein the step of performing the correction action includes one or more of the following steps: updating the design of the corresponding device component; updating the quality of the corresponding device component; updating the dimensions of the corresponding device component; updating the feature layout of the corresponding device component; updating the component manufacturing operation to produce the corresponding device component; or performing a root cause analysis to determine updates to the corresponding device component or the one or more corresponding substrate processing operations.

11. A system for data integration, comprising: a memory; and a processor coupled to the memory, configured to: receive a plurality of sets of component data associated with a substrate processing device, wherein each of the plurality of sets of component data includes a corresponding component value and a corresponding component identifier, and wherein each of the plurality of sets of component data is associated with hardware parameters of a corresponding device component of the substrate processing device; and receive a plurality of sets of sensor data, wherein each of the plurality of sets of sensor data includes a corresponding sensor value and a corresponding sensor data identifier associated with the generation of one or more corresponding substrates by the substrate processing device. The method includes receiving a plurality of sets of metrological data, each of which includes a corresponding metrological value associated with one or more corresponding substrates and a corresponding metrological data identifier; generating a plurality of aggregated sets of component sensor metrological data, each of which includes a corresponding set of device component data, a corresponding set of sensor data, and a corresponding set of metrological data; and causing a calibration action associated with the substrate processing device to be performed based on the aggregated sets of component sensor metrological data, wherein causing the calibration action includes training a machine learning model using the aggregated sets of component sensor metrological data.

12. The system as described in claim 11, wherein: The plurality of sets of component data includes one or more of component manufacturing data, component measurement data, or component material property data; and at least a portion of the plurality of sets of component data is measured by one or more of the following: an automated optical inspection (AOI) device, an atomic force microscope (AFM), or a coordinate measuring machine (CMM).

13. The system as claimed in claim 11, wherein, in order to cause the correction action to be performed, the processor is configured to provide the plurality of sets of aggregated component sensor measurement data to a trained machine learning model and receive one or more outputs from the trained machine learning model to perform the correction action.

14. The system of claim 11, wherein, in order to cause the correction action to be performed, the processor is used to store the plurality of sets of aggregated component sensor measurement data to train a machine learning model to provide a trained machine learning model, and wherein the trained machine learning model is capable of producing one or more outputs to perform the correction action.

15. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processing device, cause the processing device to: receive a plurality of sets of component data associated with a substrate processing apparatus, wherein each of the plurality of sets of component data includes a corresponding component value and a corresponding component identifier, and wherein each of the plurality of sets of component data is associated with hardware parameters of a corresponding device component of the substrate processing apparatus; and receive a plurality of sets of sensor data, wherein each of the plurality of sets of sensor data includes a corresponding sensor value and a corresponding sensor data identifier associated with the generation of one or more corresponding substrates by the substrate processing apparatus. The method includes receiving a plurality of sets of metrological data, each of which includes a corresponding metrological value associated with one or more corresponding substrates and a corresponding metrological data identifier; generating a plurality of aggregated sets of component sensor metrological data, each of which includes a corresponding set of component data, a corresponding set of sensor data, and a corresponding set of metrological data; and causing a calibration action associated with the substrate processing apparatus to be performed based on the aggregated sets of component sensor metrological data, wherein causing the calibration action includes training a machine learning model using the aggregated sets of component sensor metrological data.

16. The non-transitory computer-readable medium as described in claim 15, wherein: The plurality of sets of component data includes one or more of component manufacturing data, component measurement data, or component material property data; and at least a portion of the plurality of sets of component data is measured by one or more of the following: an automated optical inspection (AOI) device, an atomic force microscope (AFM), or a coordinate measuring machine (CMM).

17. The non-transitory computer-readable medium as described in claim 15, wherein, in order to perform the correction action, the processing means is used to provide the plurality of sets of aggregated component sensor measurement data to a trained machine learning model and receive one or more outputs from the trained machine learning model to perform the correction action.

Images