Methods and mechanisms for improving monitoring capabilities using the rate of change of sensor values

Abstract

An electronic device manufacturing system configured to acquire current sensor data associated with a sensor in a substrate manufacturing system and to determine a tilt value associated with the current sensor data. In response to determining that the tilt value meets a threshold criterion associated with the fault detection limit, at least one of the following is performed: a warning is generated or corrective action is taken.

Description

[Technical Field]

[0001] This disclosure relates to an electrical component, and more particularly to a method and mechanism for monitoring capability by using the rate of change of a sensor value. [Background technology]

[0002] The manufacture of modern materials often requires various deposition techniques, such as chemical vapor deposition (CVD) or physical vapor deposition (PVD), which deposit one or more selected types of atoms or molecules onto semiconductor devices (e.g., substrates) held in a low-vacuum or high-vacuum environment provided by a vacuum processing chamber (e.g., deposition, etching, etc.). Materials manufactured in this way can include single crystals, semiconductor films, microcoatings, and numerous other substances used in practical applications such as electronic device manufacturing. Many of these applications depend on the purity and specifications of the material grown within the processing chamber. The quality of such materials, in turn, depends on whether the manufacturing operation adheres to the correct process specifications. To maintain isolation between chamber environments and minimize exposure of substrates to the ambient atmosphere and contaminants, various sensor detection techniques are used to monitor the processing chamber environment, substrate transport, and the physical and chemical properties of the product, detecting potential anomalies and problems. Improving the accuracy, reliability, and efficiency of such monitoring presents numerous technical challenges, but successfully overcoming these challenges will facilitate the continuous progress of electronic device manufacturing and help meet the ever-increasing demands for product quality in semiconductor device manufacturing. [Overview of the project]

[0003] The following is a simplified summary of the Disclosure to provide a basic understanding of certain aspects of the Disclosure. This summary is not a comprehensive overview of the Disclosure. It is not intended to identify any major or significant elements of the Disclosure, nor to define the scope of any particular embodiment or claim of the Disclosure. Its sole purpose is to present some concepts of the Disclosure in a simplified form as a prelude to more detailed explanations to be presented later.

[0004] In one aspect of the present disclosure, an electronic device manufacturing system is configured to acquire current sensor data associated with a sensor in a substrate manufacturing system and to determine a slope value associated with the current sensor data. In response to determining that the slope value meets a threshold criterion associated with a fault detection limit, at least one of the following is performed: a warning is generated or a corrective action is taken.

[0005] In one aspect of the present disclosure, an electronic device manufacturing system is configured to acquire multiple datasets, each containing sensor output data from a plurality of sensors, each associated with a corresponding process chamber among a plurality of process chambers. The system is further configured to combine the multiple datasets into an aggregate dataset, generate a distribution of the aggregate dataset, and identify fault detection limits based on deviation values ​​generated from this distribution.

[0006] In another aspect of the present disclosure, an electronic device manufacturing system is configured by a processor to acquire output data associated with a sensor. The processor further generates a first distribution based on the output data and a set of time values, and a second distribution based on the output data and a set of tool lifetime values. The processor further generates a set of coefficients of variation based on the first and second distributions, and a set of correlation coefficients based on the first and second distributions. In response that the set of coefficients of variation meets a first threshold criterion and the correlation coefficients meet a second threshold criterion, the processor assigns the sensor to a group.

[0007] Further aspects of this disclosure include methods according to any aspect or embodiment described herein.

[0008] Further aspects of this disclosure include a non-temporary computer-readable storage medium containing instructions that, when executed by a processing unit operably coupled to memory, perform operations according to any aspect or embodiment described herein.

[0009] This disclosure is shown in the attached drawings as an example, not as an limitation. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows an exemplary system architecture according to some embodiments of the present disclosure. [Figure 2] This is a schematic top view of an exemplary manufacturing system according to some embodiments of the present disclosure. [Figure 3] This block diagram shows an exemplary predictive architecture according to some embodiments of the present disclosure. [Figure 4A] This is a series of graphs showing exemplary output values ​​of sensors classified as setpoint sensors according to some embodiments of the present disclosure. [Figure 4B] This is a series of graphs showing exemplary output values ​​of sensors classified as tool life-dependent sensors according to some embodiments of the present disclosure. [Figure 4C] This is a series of graphs showing exemplary output values ​​of sensors classified as variability sensors according to some embodiments of the present disclosure. [Figure 5A] This figure shows an example of a graphical user interface (GUI) for a health index according to some embodiments of the present disclosure. [Figure 5B] This figure shows an example of a heatmap GUI according to some embodiments of the present disclosure. [Figure 5C] This figure shows an example of a list of sensors according to some embodiments of the present disclosure. [Figure 6A]A graph showing exemplary output values from three process chambers, according to some embodiments of the present disclosure. [Figure 6B] A graph showing combined (e.g., aggregated) output values from three process chambers, within the same graph (e.g., as a single dataset), according to some embodiments of the present disclosure. [Figures 7A-7B] A graph showing how an aggregated and normalized data distribution is generated, according to some embodiments of the present disclosure. [Figure 8A] A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8B] [[ID=1)2]]A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8C] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8D] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8E] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8F] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8G] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 8H] [ A graph showing one type of control chart that can be generated for various applications, according to an aspect of the present disclosure. [Figure 9] [ A flowchart of a method for generating sensor group data, according to some embodiments of the present disclosure and in accordance with an aspect of the present disclosure. [Figure 10] A flowchart of a method for fault detection based on aggregated statistics, according to an aspect of the present disclosure. [Figure 11]A flowchart of a method for determining a projected management line according to an aspect of the present disclosure. [Figure 12] A block diagram showing a computer system according to a particular embodiment.

Mode for Carrying Out the Invention

[0011] This specification describes techniques directed to methods and mechanisms for improving monitoring capabilities using the rate of change of sensor values. The disclosed embodiments provide for handling large amounts of raw and / or statistical data from multiple sensors that supply data regarding manufacturing systems and the processes executed therein. For example, the disclosed embodiments can help accurately classify sensors into groups based on, for example, the behavior of the output values of each sensor. The embodiments can further monitor different components and subsystems across multiple process chambers and can accurately detect when an anomaly indicating a decrease in product yield occurs in the manufacturing process and / or the product of the process. A subsystem can refer to, for example, a pressure subsystem, a flow rate subsystem, a temperature subsystem, etc., and each subsystem has one or more components. Components can include, for example, a pressure pump, a vacuum, a gas supply line, and the like.

[0012] By supplying and retrieving substrates with a robot and maintaining a controlled environment in the loading chamber, processing chamber, and transfer chamber, the speed, efficiency, and quality of semiconductor device manufacturing are improved. A typical semiconductor device manufacturing process often requires dozens or hundreds of steps, such as introducing gas into the processing chamber, heating the chamber environment, changing the gas composition, purging the chamber, exhausting the gas, changing the pressure, moving the substrate from one position to another, generating or adjusting a plasma environment, performing an etching or deposition step, and so on. Since semiconductor manufacturing technology is very complex, it is necessary to process a continuous stream of runtime data from various sensors placed inside the manufacturing system. Such sensors may include temperature sensors, pressure sensors, chemical sensors, gas flow sensors, motion sensors, position sensors, optical sensors, and other types of sensors. The manufacturing system can have multiple sensors of the same (or similar) type distributed across various parts of the system. For example, a single processing chamber can have multiple chemical sensors to detect the concentration of chemical vapor at various positions within the processing chamber, and similarly, multiple temperature sensors to monitor the temperature distribution. Some or all of the sensors can output a continuous stream of data. For example, a temperature sensor can output temperature readings every second (or more frequently), so that in a single etching step that takes several minutes to execute, hundreds of data points can be generated from just this one sensor.

[0013] Each sensor (either alone or in combination with others) can output data indicating sudden or gradual adverse changes in the environment or manufacturing process settings. In addition, similar sensors in different process chambers may exhibit different behaviors (e.g., degradation rate, output fluctuations, etc.), particularly after each process chamber has undergone maintenance. In some systems, the detection system can read the data and monitor whether the manufacturing process is conforming to process specifications. However, these systems can feed all available sensor data into the detection system. When such a vast number of sensors provide data for multiple substrates processed in multiple chambers, significant variability can occur, making it difficult to identify and classify anomalies in the data obtained from a particular sensor. This can lead to the detection system generating false positives and / or false negatives, resulting in inaccurate diagnoses. Furthermore, the large datasets generated by acquiring data from hundreds or thousands of sensors can require enormous processing time. This can cause the detection system to encounter increased delays, potentially missing opportunities to make adjustments and other corrective actions during the manufacturing process, leading to substrate defects.

[0014] Furthermore, some systems perform univariate analysis to receive sensor values ​​from sensors and determine whether the sensor values ​​exceed set limits (e.g., temperature exceeding the maximum temperature, failure detection limit). Other systems perform multivariate analysis to receive sensor values ​​from multiple sensors, input the sensor values ​​into a set algorithm, receive an output, and determine whether the output exceeds a set limit (e.g., failure detection limit). Some systems may generate a failure detection management chart (e.g., showing set univariate or multivariate failure detection limits). The failure detection management chart may be used to detect failures (e.g., abnormal wafers, manufacturing equipment, etc.) and determine the cause of the failure.

[0015] Monitoring and maintaining fault detection management charts (e.g., tens of thousands of fault detection management charts) requires significant human resources. In some systems, calculating accurate fault detection limits can be time-consuming and involve trial and error (e.g., selecting which sensor values ​​to use for fault detection limits). Due to equipment aging and drift, fault detection limits generated in some systems may become obsolete. Some systems may derive set limits based only on normal products (e.g., not considering abnormal products). Some systems may not consider the interactions between sensor statistics (e.g., relationships). Some systems may not automate preventive maintenance (PM), setpoint changes, or equipment constant (EC) changes.

[0016] Aspects and embodiments of the present disclosure address these and other shortcomings of existing technologies by automatically grouping sensors and normalizing sensor output data across multiple process chambers to an automatically adaptive fault detection limit. In addition, aspects and embodiments of the present disclosure address the above and other shortcomings of existing technologies by monitoring the rate of change associated with a particular sensor value and triggering corrective actions based on whether the rate of change meets a threshold criterion. In particular, process chambers in semiconductor device manufacturing equipment can perform each substrate manufacturing process (e.g., deposition process, etching process, polishing process, etc.) according to a process policy. A process policy defines a specific set of operations performed on the substrate during the process and may include one or more settings associated with each operation. For example, a deposition process policy may include a temperature setting for the process chamber, a pressure setting for the process chamber, a flow rate setting for the precursor of the material contained in the film deposited on the substrate surface, etc. For each step of the process policy, sensors in the manufacturing equipment can generate raw sensor data related to these and other settings (e.g., measured temperature during each step and / or process execution, measured pressure during each step and / or process execution, etc.).

[0017] Embodiments of this disclosure allow for sorting or classifying one or more sensors related to a manufacturing process into one or more groups. Each group may be defined by a specific characteristic or feature of the sensor, or by the data generated by the sensor. For example, these groups may be defined based on the sensor configuration, the type of sensor output data, the quality of the sensor, or the subsystem to which the sensor is associated (e.g., a flow subsystem, a temperature subsystem, a pressure subsystem, etc.). Sensors from one or more process chambers, or from process chambers of multiple manufacturing systems, can be grouped. In one example, one or more sensors may be classified into a setpoint group, a tool life dependent group, or a variability group. A setpoint group may include sensors whose output contains, or is expected to contain, a tight distribution of data over time (e.g., little or no variation in output value over time or over the tool life, called tool life). A tool life dependent group may include sensors whose output value drifts or changes over time or over the tool life. Drift or variation may result from, for example, degradation, corrosion, or erosion of the process chamber and / or its components, variations in the coating or conditioning of the process chamber, high-frequency output time, the emissivity of the process chamber, or the lifetime of components (e.g., the lifetime of heater components). The variability group may include sensors that produce spikes in output data, or sensors that produce inconsistent or asymmetric data. In some embodiments, the variability group may include sensors that are neither part of the setpoint group nor part of the tool lifetime dependent group.

[0018] Embodiments of the present disclosure can also automatically aggregate and normalize sensor data to generate one or more sets of fault detection limits. In exemplary cases where multiple detection limits are used, a first set of control limits may reflect a “caution” limit indicating that data is outside the normal range but not within a range that could cause an anomaly to the substrate. A second set of fault detection limits may reflect a critical limit, indicating the possibility of damage occurring during the manufacturing process. To aggregate and normalize sensor data, the system of the present disclosure can combine datasets of the same sensor from different process chambers into a single dataset. The system can then generate a distribution of sensor data and identify or generate one or more sets of fault detection limits. For example, the distribution of sensor data may be a normal distribution or a Gaussian distribution. The range of each standard deviation in a Gaussian distribution can be associated with a set of fault detection limits. In particular, sensor data within a first standard deviation can be identified as normal sensor data. The first standard deviation can be set as a first upper and lower fault detection limit, and sensor data between the first and second standard deviations can be identified as “caution” sensor data. A second standard deviation can be set as a second upper and lower fault detection limit, and sensor data between the second and third standard deviations (or outside the second standard deviation) can be identified as "critical" sensor data. The limit values ​​can be determined for each substrate produced or for a specific period after maintenance operations have been performed on the process chamber. For example, for each substrate, the standard deviation of the corresponding data points can be determined, and the fault detection limit can be identified. This allows the fault detection limit to be a dynamic limit (updated, for example, for each substrate produced) rather than a static limit. Thus, the dynamic limit takes into account the degradation state associated with the continued use of the process chamber.

[0019] In some embodiments, the fault detection limit may indicate that corrective action or maintenance operation should be performed. Corrective action may include one or more operations performed to adjust the operating conditions of the process chamber (e.g., process policy parameters) during process execution. For example, corrective action may include increasing the heater current or decreasing the gas flow rate. Maintenance operation may include preventive maintenance (PM) operations, setting changes, equipment constant (EC) changes, or replacement of equipment components.

[0020] In some embodiments, the system of the Disclosure can generate a fault detection management chart. The detection management chart can be a graph used to monitor sensor data over a period of time (e.g., time, board cycle, etc.) to determine whether process variation over that period is consistent (e.g., with respect to expected limits) or abnormal (e.g., outside the detection limits). The detection management chart may include one or more sets of fault detection limits and control lines. The control lines may reflect the current sensor data over that period. In some embodiments, the rate of change of two or more sensor values ​​on the control line can be determined. The rate of change is called the “slope value” and may reflect the rate of change of control chamber wear that affects the rate of change of process drift. The slope value can be used to project the control line onto the fault detection management chart (e.g., by extrapolation). In response to determining that the projected control line intersects (or crosses) a fault detection limit, the system of the Disclosure can generate a warning. Alternatively, the system of the Disclosure can generate a warning in response to determining that the slope value itself meets a threshold criterion. In either case, it may indicate an imminent problem, and therefore the system operator (e.g., the user) can take corrective action or maintenance operations before the fault detection limit is triggered and an imminent problem occurs.

[0021] Aspects of this disclosure provide technical advantages in improving the accuracy of manufacturing fault detection techniques during the manufacturing process. For example, aspects of this disclosure can reduce the occurrence of false positives and false negatives in fault detection techniques. This allows for the generation of diagnostic data with fewer errors and inaccuracies, resulting in a reduction in the production of inconsistent and abnormal products and preventing unplanned user time or downtime. Other aspects of this disclosure can predict faults. This allows operators to take corrective actions or maintenance operations before actual faults occur, thus preventing potentially larger problems. Furthermore, aspects of this disclosure provide a significant reduction in the time and data required to process sensor data to detect potential faults and anomalies.

[0022] Figure 1 shows an exemplary computer system architecture 100 according to an aspect of the present disclosure. In some embodiments, the computer system architecture 100 may be included as part of a manufacturing system for processing substrates. The computer system architecture 100 includes a client device 110, a manufacturing apparatus 124, a prediction system 160 (e.g., for generating prediction data, providing model adaptation, using a knowledge base, etc., as described in detail in Figure 3), and a data store 140. The manufacturing apparatus 124 may include a sensor 126 configured to capture data from substrates being processed in the manufacturing system. In some embodiments, the manufacturing apparatus 124 and the sensor 126 may be part of a sensor system including a sensor server (e.g., a field service server (FSS) in a manufacturing facility) and a sensor identifier reader (e.g., a forward-opening unified pod (FOUP) radio frequency identification (RFID) reader for the sensor system). In some embodiments, a measurement apparatus may be part of the computer system architecture 100 including a measurement server (e.g., a measurement database, measurement folders, etc.) and a measurement identifier reader (e.g., a FOUP RFID reader for the measurement system).

[0023] The manufacturing apparatus 124 can produce products such as electronic devices by operating according to a policy or over a period of time. The manufacturing apparatus 124 may include a process chamber. The manufacturing apparatus 124 can perform processes on a substrate (e.g., a wafer) in the process chamber. Examples of substrate processes include a deposition process that deposits one or more layers on the surface of the substrate, and an etching process that forms a pattern on the surface of the substrate. The manufacturing apparatus 124 can perform each process according to a process policy. The process policy defines a specific set of operations performed on the substrate during the process and may include one or more settings associated with each operation. For example, a deposition process policy may include a temperature setting for the process chamber, a pressure setting for the process chamber, and a flow rate setting for the precursor material contained in the film to be deposited on the substrate surface.

[0024] In some embodiments, the manufacturing apparatus 124 includes sensors 126 configured to generate data associated with a substrate being processed in the manufacturing system 100. For example, the process chamber may include one or more sensors configured to generate spectral or non-spectral data associated with the substrate before, during, and / or after a process (e.g., a deposition process, an etching process, etc.) is performed on the substrate. In some embodiments, the spectral data generated by the sensors 126 may indicate the concentration of one or more materials deposited on the surface of the substrate. Sensors 126 configured to generate spectral data associated with the substrate may include reflectance sensors, ellipsometry sensors, thermal spectral sensors, capacitive sensors, and the like. Sensors 126 configured to generate non-spectral data associated with the substrate may include temperature sensors, pressure sensors, flow sensors, voltage sensors, and the like. For example, each sensor 126 may be a temperature sensor, pressure sensor, chemical detection sensor, chemical composition sensor, gas flow sensor, motion sensor, position sensor, optical sensor, or any other type of sensor. Some or all of the sensor 126 includes a light source for generating light (or any other electromagnetic radiation), which can be directed at a target such as a component of the machine 100 or a substrate, or a film deposited on the substrate, and the light reflected from the target can be detected. The sensor 126 can be placed anywhere within the manufacturing apparatus 124 (e.g., in any chamber including the loading station, on one or more robots, on robot blades, between chambers, etc.), or even outside the manufacturing apparatus 124 (where the sensor can inspect ambient temperature, pressure, gas concentration, etc.). Further details regarding the manufacturing apparatus 124 are provided with reference to Figure 2.

[0025] In some embodiments, sensor 126 provides sensor data (e.g., sensor values, features, trace data) associated with the manufacturing apparatus 124 (e.g., associated with the manufacturing apparatus 124 producing corresponding products such as substrates). The manufacturing apparatus 124 can produce products by performing operations according to a policy or over a period of time. Sensor data received over a period of time (e.g., corresponding to at least part of a policy or operation) may be called trace data (e.g., past trace data, current trace data, etc.) received from different sensors 126 over time. Sensor data may include one or more values ​​from among temperature (e.g., heater temperature), spacing (SP), pressure, high-frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, material flow, power, voltage, etc. Sensor data may be associated with or indicate hardware parameters such as the settings or components of the manufacturing apparatus 124 (e.g., size, type, etc.) or manufacturing parameters such as process parameters of the manufacturing apparatus 124. Sensor data may be provided while the manufacturing apparatus 124 is performing a manufacturing process (e.g., readings of the apparatus as it processes products). Sensor data may vary from board to board.

[0026] In some embodiments, the manufacturing apparatus 124 may include a control device 125. The control device 125 may include one or more components or subsystems configured to enable and / or control one or more processes of the manufacturing apparatus 124. For example, subsystems may include a pressure subsystem, a flow subsystem, a temperature subsystem, and so on, and each subsystem may have one or more components. Components may include, for example, a pressure pump, a vacuum, a gas supply line, a plasma etching apparatus, an actuator, and so on. In some embodiments, the control device 125 may be managed based on data from a sensor 126.

[0027] In some embodiments, a particular sensor 126 and control device 125 may be associated with one or more control modules. In particular, each control module may include a set of sensors such as 126, control device 125, and control logic that modulates the sensors and / or components. In exemplary examples, control modules may include a thermal control module, a plasma control module, a reactant flux control module, and a substrate control module. A thermal control module may include sensors and control devices (e.g., heaters, heater sensors, etc.) related to providing and maintaining a heated environment within the process chamber. A plasma control module may include sensors and control devices (e.g., plasma etchers, etcher sensors, etc.) related to generating or regulating the plasma environment within the process chamber. A reactant flux control module may include sensors and control devices (e.g., gas flow control devices and sensors, pumps, etc.) related to gas flow operations within the process chamber. A substrate control module may include sensors and control devices related to substrate characteristics (e.g., warping of the substrate). In certain embodiments, sensor data from one or more of a particular control module can be processed and analyzed via modules 111-117 and the methods discussed herein to control the respective operating conditions (e.g., parameters of a process policy) associated with the process control module.

[0028] The client device 110 may include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbooks, network-connected televisions ("smart TVs"), network-connected media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, and operator boxes. In some embodiments, sensor data (or other data items) can be received from the client device 110. The client device 110 may display a graphical user interface (GUI), which allows the user to provide measurement values ​​of substrates processed in the manufacturing system as input. The client device 110 may include a sensor control module (SCM) 111, a sensor statistics module (SSM) 112, a grouping module 113, a monitoring module 114, a fault detection module (FDM) 115, a chart generator 116, and a corrective action module 117.

[0029] The SCM111 can activate, deactivate, idle, change sensor settings, and detect sensor hardware or software problems. In some embodiments, the SCM111 can track processing operations performed by the manufacturing equipment 124 and determine which sensors 126 should be sampled for specific processing (or diagnostic, maintenance, etc.) operations of the manufacturing equipment 124. For example, during a chemical deposition step inside one of the processing chambers, the SCM111 can sample sensors 126 located inside each processing chamber, but cannot activate (or sample) sensors 126 located inside the transfer chamber and / or loading station. The raw data acquired by the SCM111 may include time-series data that captures or generates one or more readings of the amount detected by a particular sensor 126 over a series of time instances t1, t2, ... t

[0030] In which, N pressure readings P(t i ) can be generated. In some embodiments, the raw data acquired by the SCM 111 can include a spatial map at a predetermined series of spatial positions. For example, an optical reflectivity sensor can measure the reflectivity R(x j , y l ) of a film deposited on the surface of a wafer at a set of spatial positions x j , y k (e.g., a two-dimensional set). In some embodiments, both time-series and spatial map raw data can be collected. For example, while depositing a film on a wafer, the SCM 111 can collect reflectivity data R(t i [[ID=1​​​​​​​The SSM112 can process the raw data acquired from the sensor 126 by the SCM111 and determine statistical values ​​(referred to as “statistical data”) that represent that raw data. For example, for each or part of the raw sensor data distribution, the SSM112 can determine one or more parameters of the distribution, such as the mean, median, mode, upper bound, lower bound, variance (or standard deviation), skewness (third moment), kurtosis (fourth moment), or any further moments or cumulant of the data distribution. In some embodiments, the SSM112 can model the raw data with various model distributions (normal distribution, log-normal distribution, binomial distribution, Poisson distribution, gamma distribution, or any other distribution) (e.g., via regression analysis fitting). In such embodiments, one or more parameters may include fitting parameters determined by the SSM112, along with identification information of the fitting distribution being used. In some embodiments, the SSM112 can use multiple distributions, such as a principal distribution and tail distributions for outlier data points, to fit raw data from a single sensor. The parameters of the distribution obtained by the SSM112 can be sensor-specific. For example, some sensors may allow for the determination of a small number of parameters (mean, median, variance), while others may allow for the determination of many more (e.g., 10 or 20) moments.

[0031] The grouping module 113 can be configured to sort or categorize one or more sensors related to a manufacturing process or manufacturing apparatus 124 into one or more groups. In some embodiments, the grouping module can use sensor data obtained from SCM 111, SSM 112, or any other module or data store containing raw or ongoing sensor data. Each group can be defined by specific characteristics or features of the sensor or the data generated by the sensor. For example, these groups can be defined based on the sensor configuration, the type of sensor output data, the sensor quality, the subsystem to which the sensor is associated (e.g., flow subsystem, temperature subsystem, pressure subsystem, etc.). Sensors from one or more process chambers of the manufacturing apparatus 124, or from process chambers of multiple manufacturing apparatuses, can be grouped.

[0032] In some embodiments, one or more sensors can be categorized into setpoint groups, tool life-dependent groups, or variability groups. Setpoint groups may include sensors whose output includes, or is expected to include, a tight distribution of data over time (e.g., little or no variation in output values ​​over time or over the tool life, referred to as tool life). For example, setpoint sensors may include temperature sensors, high-frequency power sensors, gas flow sensors, and the like. In some embodiments, the tight distribution of data can be correlated with the expected results of a process policy. For example, during a manufacturing process, the process chamber temperature may be expected to be constant. Thus, deviations from the expected value or deviations from a predefined limit of the expected value can be flagged as a fault by, for example, the fault detection module 115.

[0033] In an exemplary example, Figure 4A is a series of graphs showing exemplary output values ​​of a sensor classified as a setpoint sensor according to an aspect of the present disclosure. In particular, column 410 is a series of three graphs showing the output values ​​of process chamber flow rates for three different process chambers. The y-axis represents the flow rate value, and the x-axis represents the number of process runs on the substrate. Column 420 is also a series of three graphs showing the output values ​​of process chamber flow rates for three different process chambers. The y-axis represents the flow rate value, and the x-axis represents time. As shown by the series of graphs shown in columns 410 and 420, the flow rate values ​​are consistent over time and the number of substrates.

[0034] Returning to Figure 1, in some embodiments, one or more sensors can be categorized into a tool life-dependent group. The tool life-dependent group may include sensors whose output values ​​drift or change over time or during the tool's life. This drift or change may result from, for example, degradation, corrosion, or erosion of the process chamber and / or its components, variations in the process chamber's coating or conditioning, high-frequency output time, process chamber emissivity, or component lifespan (e.g., heater component lifespan). In one example, sensors categorized into the tool life-dependent group may include heater output value, foreline pressure, high-frequency impedance, etching rate, and so on.

[0035] In an exemplary example, Figure 4B is a series of graphs showing exemplary output values ​​of a sensor classified as a tool life-dependent sensor according to an aspect of the present disclosure. In particular, column 430 is a series of three graphs showing output values ​​of heater output power for three different process chambers. The y-axis represents the percentage value of heater output power, and the x-axis represents the number of process runs on the substrate. Column 440 is also a series of three graphs showing output values ​​of heater output power for three different process chambers. The y-axis represents the percentage value of heater output power, and the x-axis represents time. As shown by the series of graphs shown in columns 430 and 440, the percentage value of heater output power drifts consistently over time and the number of substrates.

[0036] Returning to Figure 1, in some embodiments, one or more sensors can be classified into a variability group. The variability group may include sensors that have spikes in their output data, sensors that produce inconsistent or asymmetric data, and so on. In some embodiments, the variability group may include sensors that are neither part of the setpoint group (and therefore do not have a tight distribution) nor part of the tool life dependent group (and therefore do not experience drift correlated with time or tool life). In some examples, sensors classified into the variability group may include reflected power sensors, back-side flow sensors, sensors that produce relatively large amounts of noise, sensors whose output values ​​change over time but do not correlate with tool life, and sensors that do not affect the tool.

[0037] In an exemplary example, Figure 4C is a series of graphs showing exemplary output values ​​of a sensor classified as a variability sensor according to an aspect of the present disclosure. In particular, column 450 is a series of three graphs showing output values ​​of process chamber temperature for three different process chambers. The y-axis represents the average temperature value, and the x-axis represents the number of process runs on the substrate. Column 460 is also a series of three graphs showing output values ​​of process chamber temperature for three different process chambers. The y-axis represents the average temperature value, and the x-axis represents time. As shown by the series of graphs shown in columns 450 and 460, the average temperature value includes many values ​​that deviate from the average and may be due to noise.

[0038] Returning to Figure 1, the embodiments of this disclosure are discussed in relation to sensor groups, but in some embodiments, the grouping module 113 can be configured to organize one or more data items into one or more groups. The data items may include sensor data, task data, context data, statistical data, etc. In some embodiments, these data items can first be grouped into a set of “arrays”. An array may include combinations of data items according to a predefined format or pattern. In some embodiments, each array may include a specific sensor (e.g., chamber pressure sensor, heater current sensor, etc.), a statistical data type (e.g., mean, range, etc.), and a policy part identifier (e.g., step 1, step 5, entire policy, etc.). For example, an array may show the average heater voltage during step 3 of a particular process policy.

[0039] In some embodiments, the grouping module 113 can classify sensors into one or more specific groups using one or more algorithms. In some embodiments, the grouping module 113 can classify sensors into one or more specific groups using a detection algorithm. The detection algorithm can be configured to associate each sensor with one or more predefined groups based on one or more predefined criteria. In one embodiment, the detection algorithm can first perform a distribution algorithm (e.g., a Gaussian distribution algorithm, or any other algorithm or formula that can determine deviations in a dataset) to generate a distribution for each sensor based on time and tool lifetime.

[0040] The detection algorithm can then calculate the coefficient of variation and correlation coefficient for both distributions (e.g., a time-based distribution and a tool-life-based distribution). The coefficient of variation (CV) is the ratio of the standard deviation to the mean and indicates the degree of variation relative to the population mean. In one example, the coefficient of variation can be determined by dividing the population standard deviation by the population mean. The correlation coefficient is a number between -1 and 1 that indicates the strength and direction of the relationship between variables (e.g., a statistical relationship between two variables). In particular, the correlation algorithm can be configured to identify the pairing between corresponding output values ​​of both distributions.

[0041] In some embodiments, the correlation algorithm may include a clustering algorithm that can receive corresponding sensor values ​​from both distributions as input data and generate an index of correlation as output data. In some embodiments, the grouping module 113 may generate output data using, for example, a clustering algorithm. The clustering algorithm may include a K-means clustering algorithm, a density-based spatial clustering (SBSCAN) algorithm for applications with noise, a spectral clustering algorithm, a Ward clustering algorithm, a Birch clustering algorithm, or any other clustering algorithm.

[0042] In response to the coefficient of variation meeting a setpoint criterion (for example, the coefficient of variation is below a threshold and the correlation is within a specific range for a value of 1), the grouping module 113 can classify those sensors as setpoint sensors. In response to the coefficient of variation meeting a tool life dependency criterion (for example, the coefficient of variation exceeds a threshold and the correlation of tool life data across multiple process chambers is within a specific range for a value of 1), the grouping module 113 can classify those sensors as tool life dependency sensors. In response to neither the setpoint criterion nor the tool life dependency criterion being met, the grouping module 113 can classify those sensors as variability sensors. Further details regarding the distribution algorithm are described in Figure 9.

[0043] In some embodiments, the grouping module 113 can classify sensors into one or more specific groups using one or more machine learning models (e.g., model 190). In particular, the grouping module 113 can input sensor data (e.g., sensor values, sensor characteristic data, etc.) into a machine learning model and receive data as output indicating group assignments (e.g., which group each sensor should be assigned to). The machine learning model can be generated by the prediction system 160 described with reference to Figure 3.

[0044] In some embodiments, the grouping module 113 can use an automated sensor ranking algorithm. The automated sensor ranking algorithm can group sensors based on the importance or significance of each sensor. The importance or significance of each sensor can be determined by comparing values ​​from similar or the same sensor, such as from different process chambers, during different process executions, and from good process executions (e.g., when the substrate is manufactured accurately or close to the desired specifications, compared to when the substrate is manufactured with defects or deformation). In an exemplary example, the grouping module 113 can monitor a series of process executions of a process policy and collect runtime data from a series of sensors on the manufacturing apparatus 124. The grouping module 113 can determine qualitative data describing each substrate produced by the series of process executions of the process policy. Based on the analysis of the qualitative data, the grouping module 113 can characterize each process execution into its respective predetermined group. Next, the grouping module 113 can generate a data model based on the collected runtime data, which describes, for each of the multiple groups, at least one pattern of sensor data for each group, and / or the relative importance of each of the set of sensors and the set of sensor types in relation to each group. In some embodiments, the grouping module 113 can classify at least one subsequent run into one of the groups by performing a multivariate analysis on additional runtime data collected during at least one subsequent run of the policy in the manufacturing environment to determine which pattern of sensor data specified in the data model best fits that additional runtime data. Once at least one subsequent run of the policy has been classified into a specific group, the grouping module 113 can generate an output (for example, for display on the interface of the client device 110) showing a ranking of at least two sensor types based on the additional runtime data and the description of the relative importance of each of the multiple sensor types for the specific group in the data model.Further details regarding the automated sensor ranking algorithm are described in U.S. Patent No. 11,054,815, which is incorporated herein by reference in its entirety.

[0045] The monitoring module 114 can generate one or more graphical user interfaces (GUIs) for monitoring one or more sensor groups. In some embodiments, the monitoring module 114 can generate a health index GUI configured to track output data generated by sensors in one or more sensor groups. The health index GUI can display each sensor (or a subset of sensors from one or more sensor groups) and their respective output values ​​over a timeline (e.g., time, number of process runs, etc.). In some embodiments, one or more limits can be displayed for one or more of the displayed sensors. Limits can indicate deviations, failures, anomalies, or any other abnormal or irregular data. In some embodiments, limits may be associated with failure detection limits, which are discussed below. In an exemplary example, Figure 5A shows an example of a health index GUI 500 according to an aspect of the present disclosure. Each listed sensor 510 (e.g., a set of sensors assigned to a tool life dependent group) includes a corresponding set of output parameters or values ​​520 displayed on the y-axis. The x-axis displays post-maintenance process runs. For some sensors, visual limits can be displayed (e.g., for lid power, foreline pressure, and high-frequency shunt impedance). For example, limit 530 displays the acceptable impedance value of the process chamber. Limit values ​​can be set, for example, via user input. If a sensor value exceeds a limit value, the monitoring module 114 can generate a warning (e.g., display or send a prompt, generate a sound, etc.). In some embodiments, corrective actions can be taken via the corrective action module 1116 in response to a sensor value exceeding a limit value.

[0046] In some embodiments, the monitoring module 114 can generate a heatmap configured to track output data generated by sensors from one or more sensor groups. Figure 5B shows an example of a heatmap GUI 540 according to an aspect of the present disclosure. In some embodiments, the y-axis displays a predetermined series of sensor output data over a period of time (e.g., a series of days represented by the x-axis). Sensors listed on the y-axis may belong to a specific group (e.g., a setpoint group, a tool life group, etc.). In some embodiments, each list on the y-axis may show failure indicator data for a particular sensor from a series of process chambers. For example, the first row shown in GUI 540 may show failure indicator data for sensor A in each process chamber using any type of process policy. The failure indicator data may, in one example, indicate whether the data output from each of the particular sensors is within the normal range (indicated by shading 542), at or near the failure limit (indicated by shading 544), or above the failure limit (indicated by shading 546). In some embodiments, if a predetermined number of specific sensors meet a threshold criterion (e.g., are at or exceed the failure limit), the failure indicator data can be visually represented by changing the color or shading. For example, failure indicator 544 indicates that at least one sensor B has exceeded the failure limit. In some embodiments, a list of all sensors (e.g., each sensor A in each process chamber) can be displayed (e.g., via user input) by selecting a failure indicator (e.g., failure indicator 542). Figure 5C shows an example of a list of each sensor. As shown, a failure has occurred in process chamber D.

[0047] The fault detection module 115 can process, aggregate, and analyze sensor data collected by the SCM module 111, statistical data collected by the SSM 112 and / or grouping module 113. In particular, the fault detection module 115 can automatically aggregate and normalize sensor data to generate fault detection limits. Fault detection limits can be indicators that the sensor output data indicates a fault or anomaly. In some embodiments, multiple fault detection limits can be used. For example, a first fault detection limit may reflect a “caution” limit (or fault), indicating that the data is outside the normal range but not within a range that could cause an anomaly to the substrate. A second fault detection limit may reflect a fatal limit (or fault), indicating that damage may have occurred during the manufacturing process.

[0048] In some embodiments, data from specific types of sensors (from one or more process chambers) can be aggregated and normalized. For example, heater output data can be acquired, aggregated, and normalized for each process chamber. In some embodiments, the fault detection module 115 can aggregate and normalize data for each type of sensor categorized into groups.

[0049] In some embodiments, the fault detection module 115 can combine different data sets into a single data set in order to aggregate and normalize sensor data. In an exemplary example, Figure 6A is a graph 610 showing exemplary output values ​​from three process chambers according to an aspect of the present disclosure. In particular, Figure 6A shows the percentage average of the respective heater output power across a number of substrates produced for three different process chambers. Figure 6B is a graph 620 showing combined (e.g., aggregated) output values ​​from three process chambers on the same graph (e.g., as a single data set) according to an aspect of the present disclosure.

[0050] The fault detection module 115 can further generate a distribution of sensor data and identify or generate one or more fault detection limits. For example, the distribution of sensor data can be a normal distribution or a Gaussian distribution. The fault detection module 115 can then identify a normal data range and set one or more fault detection limits. In one example, the fault detection limits can be based on the standard deviation of the data distribution. For example, the fault detection module 115 can determine the mean of the aggregated dataset, identify sensor output values ​​within a first standard deviation of the mean, and identify sensor values ​​between the first and second standard deviations. Each standard deviation range can be associated with a fault detection limit. For example, sensor data within the first standard deviation can be identified as normal sensor data. The first standard deviation can be set as the first fault detection limit, and sensor data between the first and second standard deviations can be identified as "caution" sensor data. A second standard deviation can be set as the second fault detection limit, and sensor data between the second and third standard deviations (or outside the second standard deviation) can be identified as "critical" sensor data. In some embodiments, the fault detection limit can be set using a training set of data. In some embodiments, the fault detection limit can be set and / or adjusted using real-time data.

[0051] Figures 7A and 7B are graphs 710 and 720, respectively, illustrating how aggregated and normalized data distributions are generated according to embodiments of this disclosure. In particular, Figure 7A shows the standard distribution of data represented by a bell curve (e.g., heater output data described in Figures 6A and 6B). Figure 7B shows a detection management chart with fault detection limits (determined from the standard deviation of the sensor data) superimposed on sensor data. In particular, the first standard deviation can correspond to a first set of fault detection limits 722 (e.g., caution fault detection limits), and the second standard deviation can correspond to a second set of fault detection limits 724 (e.g., fatal fault detection limits). Limits can be determined for each board produced (e.g., values ​​along the x-axis). For example, for each board, the standard deviation of the corresponding data point can be determined, and the fault detection limit can be identified. This allows the fault detection limit to be a dynamic limit (e.g., updated for each board produced) rather than a static limit. In some embodiments, the time after zero on the x-axis represented the period after maintenance of the process chamber. Therefore, the dynamic limits take into account the degradation associated with the continuous use of the process chamber.

[0052] Returning to Figure 1, in some embodiments, the fault detection limit can be determined using training data that includes an ideal or near-ideal process run (e.g., a process run without abnormalities). In some embodiments, the group can be updated based on data that triggers the fault detection limit. For example, during a production run, the fault detection module may determine that a particular sensor has deviated beyond a specific fault detection limit (e.g., a critical limit). These sensors are then categorized as setpoint sensors and added to the setpoint group.

[0053] It should be noted that the examples of sensor data aggregation and normalization are used as illustrative examples only, and other methods may be used. In some embodiments, the fault detection module 115 may perform preprocessing, dimensionality reduction of sensor statistics, processing of reduced statistical representations, normalization, and / or processing using neural networks, etc., to determine the fault detection limit. At least some of the listed operations may involve machine learning.

[0054] The chart generator 116 can generate a fault detection management chart. A fault detection management chart can be any type of chart, graph, plot, or other visual representation used to display (e.g., on the GUI of the client device 110) the monitoring of sensor data over a period of time (e.g., time, board cycle, etc.) and to determine whether process fluctuations over that period are consistent (e.g., within expected limits) or abnormal (e.g., outside detection limits). A fault detection management chart can include one or more sets of fault detection limits and control lines. Each set of fault detection limits can include an upper fault detection limit and a lower fault detection limit. Control lines can reflect the current sensor data over that period. Therefore, the chart generator 116 can generate a fault detection management chart by first providing (e.g., drawing or plotting) the fault detection limits (generated using the fault detection module 115) on a graph. In response to receiving corresponding current sensor data, the chart generator can plot that sensor data on the detection management chart. Two or more sensor data points can be called control lines. As illustrated in detail in Figures 8A to 8H, different types of management charts can be generated for various applications. These examples are illustrative and not intended to be limiting.

[0055] Figure 8A is a diagram of a detection control chart 800 used in connection with a chamber seasoning operation according to an aspect of the present disclosure. Preventive maintenance can be performed periodically on the process chamber. Preventive maintenance may include cleaning the process chamber, repairing process chamber components, performing adjustment or tuning of components, replacing one or more components, and / or any other procedures that can be performed on process chamber components to reduce the likelihood of equipment failure. After preventive maintenance has been performed, the process chamber (starting at t=0) may experience a conditioning phase. During the conditioning phase, process runs can be performed on a series of substrates until the process chamber reaches a steady-state phase (a phase in which settings associated with the process strategy (e.g., temperature settings, pressure settings, flow rate settings, etc.) are constant or expected to change at a constant rate). The conditioning phase may be a seasoning phase in which a layer of material (e.g., a silicon oxide layer) is built on the chamber walls before the substrate is introduced into the chamber for processing. The deposited seasoning layer may result from the gases used in the process strategy and can reduce the likelihood of contaminants interfering with subsequent processing steps. During the construction of the seasoning layer, each subsequent substrate may require different settings to achieve the desired result. For example, the heater power may need to be slightly increased for each subsequent substrate because the growing seasoning layer absorbs more heat. In one example, the seasoning phase may involve 50, 100, or any other number of substrates, and this continues until the subsequent settings are no longer affected by the process execution on the subsequent substrates, at which point the process chamber reaches a steady state.

[0056] As shown in Figure 8A, the fault detection management chart 800 includes a set of fault detection limits (upper fault detection limit 812 and lower fault detection limit 814), a control line 816, and a transition line 818. The x-axis can represent a period (e.g., time, board cycle, etc.), and the y-axis can represent a sensor value (e.g., heater power, RF power, heater current, pressure, flow rate, emissivity, etc.). This period can represent a production run. The production run can reflect the number of boards processed in the process chamber since the last maintenance operation was performed. The set of control limits can be associated with the setting parameters being monitored. The transition line 818 can indicate different phases in which sensor data related to a specific setting parameter is collected. As shown in the figure, the left side of the transition line 818 (Phase 1) can include sensor data collected while the process chamber is being seasoned, and the right side of the transition line 818 (Phase 2) can include sensor data collected when the process chamber is in a steady state. Since the steady-state phase of the detection control chart 800 is expected to include a tight distribution of data over time (e.g., little or no variation in output values ​​over time or tool lifetime), the detection control chart 800 can be used for sensors in setpoint groups.

[0057] The control line reflects the collected sensor data. In some embodiments, the control line can be generated by determining the best-fit line of the sensor data (or by any other method capable of determining the relationship between the sensor data). The best-fit line is a straight line that minimizes the distance between that line and some data. The best-fit line can be determined using the least squares method or any other formula, equation, or method. The control line can be updated at predetermined intervals (e.g., according to each currently plotted sensor value, according to each currently plotted second sensor line value, etc.). As shown in Figure 8A, the control line remains within the control limits throughout the seasoning and steady-state phases. Therefore, no failure occurs.

[0058] The chart generator 116 can acquire current sensor data and plot it on a detection management chart. In some embodiments, the chart generator 116 (or fault detection module 115) can determine the y-intercept and / or slope of the control line. The y-intercept may be an initial value obtained at t=0, or it may be determined by determining the slope value of two or more sensor values ​​on the control line. In some embodiments, the slope value can be determined using the slope equation y=m*x+b, where m is the slope value, b is the y-intercept, x refers to the x-axis value, and y refers to the y-axis value. In some embodiments, the chart generator can indicate that a fault has occurred in response to the y-intercept value meeting a threshold criterion (e.g., being outside the set of control limits). For example, the chart generator 116 can plot the first sensor value received at x=0 with the appropriate y-coordinate. Next, the chart generator 116 can determine if the initial sensor value (e.g., the y-intercept value) is above the upper fault detection limit or below the lower fault detection limit. In response, the chart generator 116 can determine that a fault has occurred, issue a warning, and / or take corrective action.

[0059] In some embodiments, the chart generator 116 can project control lines (e.g., using slope values, extrapolation, or any other formula or method) and determine whether the projected control lines meet threshold criteria (e.g., are outside a set of fault detection limits). In response to determining that the projected control lines meet threshold criteria, the chart generator 116 can indicate that a fault has occurred, issue a warning, and / or take corrective action.

[0060] To generate projected control lines, the chart generator 116 can first determine the slope value of the control lines. Then, the chart generator 116 can graph the projected sensor data points based on the slope value. The projected sensor data points can be graphed over a predetermined period (e.g., 50 subsequent boards, the length of the fault detection limit or a portion thereof). The projected control lines can be regraphed at predetermined intervals (e.g., every five current data points are plotted, every time a current data point is plotted, etc.).

[0061] In some embodiments, projected control lines can be used as a visual aid to the operator. In some embodiments, the chart generator 116 can determine whether a fault detection limit is crossed by determining whether the control line intersects with one of the fault detection limits. For example, the control line can be expressed as y = m1*x + b1, where m1 is the slope of the control line and b1 is the y-intercept of the control line. The fault detection line can be expressed as y = m2*x + b2, where m2 is the slope of the fault detection line and b2 is the y-intercept of the fault detection line. The chart generator 116 can then determine the x-values ​​that reflect the intersection by setting each y-value equal to the others. The chart generator 116 can then determine whether the determined x-values ​​meet a threshold criterion. The threshold criterion may reflect, for example, the end of a phase (e.g., the end of a seasoning phase). In response to determining that the x-value is before the end of a phase, a warning can be generated.

[0062] In some embodiments, the chart generator 116 can determine upper and lower slope limits based on the slopes of the respective fault detection limits, phase indicators, and control line slopes. For example, the chart generator 116 can set the x-value to the intersection of one of the fault detection limits (e.g., the upper fault detection limit) and the phase indicator. The chart generator 116 can then determine a trigger slope value for the control line that intersects this value (based on the y-intercept value). The chart generator 116 can then trigger a warning in response to determining that the subsequently determined control line slope value for the process chamber exceeds the trigger control value. A trigger slope value can also be determined for the lower fault detection limit, and the chart generator 116 can trigger a warning in response to determining that the subsequently determined control line slope value for the process chamber falls below this trigger control value.

[0063] Figure 8B is a diagram of a fault detection control chart 802 used in relation to a sensor whose output value drifts or changes over time or during the tool's lifespan, according to an aspect of the present disclosure. In some embodiments, the detection control chart 802 can be used in relation to a group of sensors that depend on tool life, which may be caused by erosion, continuous changes in parameters (e.g., an increase / decrease in pressure in a fluid supply system due to fluid depletion), ampoule life, etc. The fault detection control chart 802 includes a set of fault detection limits (upper fault detection limit 822 and lower fault detection limit 824) and a control line 826. The x-axis may represent time, and the y-axis may represent a set parameter.

[0064] Figure 8C is a diagram of a fault detection control chart 804 used in connection with periodic cleaning of a process chamber according to an aspect of the present disclosure. Cleaning of the process chamber or its components may be required due to the accumulation of contaminants, deterioration of seasoning, etc. The fault detection control chart 804 includes a set of fault detection limits (upper fault detection limits 832A-832C and lower fault detection limits 834A-834C), periodic cleaning indicators 838A-838B, and control lines 836A-836C. The x-axis may represent time, and the y-axis may represent sensor values. As shown, a certain number of substrates can be processed until processing is interrupted for cleaning of the process chamber. After cleaning, the fault detection limits are reset. Warnings may be issued or corrective actions may be taken in response to the projected control lines triggering a fault detection limit, or the value of the y-intercept triggering a fault detection limit. For example, a warning may indicate to the operator that cleaning should be performed earlier than planned, thus preventing the production of substandard substrates.

[0065] Figure 8D is a diagram of a fault detection control chart 806 used in relation to component damage according to an aspect of the present disclosure. A process chamber may have damage to one or more components that requires replacement of those components. Damage may result from normal wear and tear. The fault detection control chart 806 includes a set of fault detection limits (upper fault detection limit 842 and lower fault detection limit 844), a steady-state indicator 840, a control line 846, and an end-of-life indicator 848. The x-axis may represent time, and the y-axis may represent sensor values. As shown, the steady-state phase begins with the steady-state indicator 840. A certain number of boards may be processed until the end-of-life indicator 848 is reached, at which point the slope of the fault detection limit is changed to account for one or more degraded components. The slope of the control line 846 can be measured periodically, and the projected control line can be determined. Before or after the end-of-life indicator 848, a warning may be issued or corrective action may be taken in response to the projected control line triggering a fault detection limit. For example, the warning may indicate to the operator that the component is degrading faster than expected and should be replaced sooner than planned.

[0066] Figure 8E is a graph 808 showing a detection control chart having multiple sets of control limits according to an aspect of the present disclosure. As shown, the detection control chart 805 includes two sets of fault detection limits (attention faults 851 and fatal faults 852) and a phase shift indicator 857. Using sensor data acquired for process chamber A 853, a control line 855 with a slope value of 0.8 can be determined. Using sensor data acquired for process chamber B 854, a control line 856 with a slope value of 2.1 can be determined. As shown, the sensor data generated by process chamber A indicates that there is a fault in that process chamber because the slope crosses multiple fault detection limits.

[0067] Figure 8F is a diagram of a fault detection control chart 806 that generates a projected control line according to an aspect of the present disclosure. The fault detection control chart 806 includes a set of fault detection limits (upper fault detection limit 862 and lower fault detection limit 864), a phase shift indicator 868, a control line 866, and a projected control line 867. In this embodiment, the left side of the phase shift indicator 868 can point to the seasoning phase, and the right side of the phase shift indicator 868 can point to the steady-state phase. The x-axis can represent a period, and the y-axis can represent a sensor value. The current sensor value can be plotted (not shown), and a control line 866 can be generated. The slope value of the control line 866 can be determined, and a projected control line 867 can be generated. The projected control line crosses the upper fault detection limit 862, indicating that a fault will occur during the seasoning process. This allows for the generation of fault indicators such as warnings. This allows the operator to be informed of the approximate timing of a fault, and thus the operator can take preventive corrective actions.

[0068] Figure 8G is another diagram of a fault detection control chart 807 that generates a projected control line according to an aspect of the present disclosure. The fault detection control chart 807 includes a set of fault detection limits (upper fault detection limit 872 and lower fault detection limit 874), a phase shift indicator 878, a control line 876, and a projected control line 877. In this embodiment, the left side of the phase shift indicator 878 can point to the seasoning phase, and the right side of the phase shift indicator 878 can point to the steady state phase. The x-axis can represent a period of time, and the y-axis can represent a sensor value. The current sensor value can be plotted (not shown) and a control line 876 can be generated. As shown, the control line 876 remained within the range of fault detection limits 872 and 874 during the seasoning phase. The slope value of the control line 876 in the steady state phase can be determined and a projected control line 877 can be generated. The projected control line crosses the lower fault detection limit 874. Thus, an indicator of fault, such as a warning, can be generated. This allows operators to be notified of the approximate timing of a malfunction, and thus they can take preventative corrective actions.

[0069] Figure 8G is another diagram of a fault detection control chart 808 according to an aspect of the present disclosure, which generates a fault due to a y-intercept value. The fault detection control chart 808 includes a set of fault detection limits (upper fault detection limit 882 and lower fault detection limit 884), a phase shift indicator 888, and a control line 887. The x-axis can represent time, and the y-axis can represent sensor values. The current sensor value can be plotted (not shown) and a control line 886 can be generated. As shown, the control line 886 has a y-intercept outside the fault detection limits. Thus, an indicator of fault, such as a warning, can be generated. This allows the operator to be notified that a fault has occurred at the start of the seasoning phase, and thus the operator can take preventive corrective action.

[0070] The corrective action module 117 can receive user input of indicators associated with the manufacturing equipment 124 (for example, via a graphical user interface (GUI) displayed via the client device 110). In some embodiments, the corrective action module 117 receives input data from the fault detection module 115 and / or the chart generator 116, determines corrective actions based on the input data, and implements the corrective actions. For example, in response to receiving an indicator that sensor data has met a threshold criterion (e.g., exceeded or fell below the fault detection limit), the corrective action module 116 may implement one or more corrective actions (e.g., increasing power, decreasing flow rate, etc.). The corrective actions can be stored in a fault pattern library on the data store 140. In some embodiments, the corrective action module 117 receives indicators for corrective actions from the prediction system 160 and implements the corrective actions. Each client device 110 may include an operating system that allows the user to generate, view, or edit data (e.g., metrics associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0071] Although shown as modules of client device 110, each module 111-117 can be included in one or more other computing devices such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, GPUs, and ASICs. Each module 111-117 can execute instructions to perform one or more of the methods and / or embodiments described herein. Instructions can be stored in computer-readable storage media, which may include main memory, static memory, secondary storage, and / or processing units (during instruction execution).

[0072] The datastore 140 can be memory (e.g., random access memory), drives (e.g., hard drives, flash drives), a database system, or another type of component or device capable of storing data. The datastore 140 may include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers). The datastore 140 can store data associated with the processing of substrates in the manufacturing equipment 124. For example, the datastore 140 can store data (referred to as process data) collected by the sensors 126 of the manufacturing equipment 124 before, during, or after the substrate process. The process data may refer to historical process data (e.g., process data generated for previous substrates processed in the manufacturing system) and / or current process data (e.g., process data generated for the current substrate processed in the manufacturing system). The datastore may also store spectral data or non-spectral data associated with a portion of the substrates processed in the manufacturing equipment 124. The spectral data may include historical spectral data and / or current spectral data.

[0073] The data store 140 can also store context data associated with one or more substrates processed by the manufacturing system. Context data may include policy names, policy step numbers, preventive maintenance indicators, operators, etc. Context data may refer to historical context data (e.g., context data associated with previous processes performed on previous substrates) and / or current process data (e.g., context data associated with current or future processes performed on previous substrates). Context data may further include identifying sensors associated with specific subsystems of the process chamber.

[0074] The data store 140 can also store task data. Task data may include one or more sets of operations performed on a substrate during a deposition process, and may include one or more settings associated with each operation. For example, task data for a deposition process may include the temperature setting of the process chamber, the pressure setting of the process chamber, and the flow rate setting of the precursor material for the film to be deposited on the substrate. In another example, task data may include controlling the pressure at a pressure point defined for a flow rate value. Task data may refer to past task data (e.g., task data associated with a previous process performed on a previous substrate) and / or current task data (e.g., task data associated with a current or future process to be performed on a substrate).

[0075] In some embodiments, the data store 140 can store statistical data. Statistical data may include statistical values ​​representative of the raw data generated by the SSM112, such as mean data, range data, standard deviation data, maximum and minimum data, median data, and mode data. Mean data may include the measured mean of two or more values. For example, mean data can be used to determine the average heater temperature, process chamber pressure, or average gas flow rate over a step, a specific duration, or an entire process strategy. Range data may include the central observation of a set of data (e.g., the median temperature during a step). Range data may also include the difference between the maximum and minimum values ​​of a set of values ​​(e.g., the range of heater pressure during a process strategy). Standard deviation is a measure of the amount of variability or dispersion of a set of values.

[0076] In some embodiments, the data store 140 can store sensor group data. Sensor group data may include data that identifies which group a sensor is assigned to. For example, a first set of sensors or arrays may be assigned to a setpoint group (by the grouping module 113), and a second set of sensors or arrays may be assigned to a tool life dependent group. In some embodiments, the sensor group data may include metadata associated with each specific sensor. In some embodiments, the sensor group data may include a data structure such as a data table that stores records, each record containing a sensor identifier and a group identifier.

[0077] In some embodiments, the data store 140 can be configured to store data that is inaccessible to users of the manufacturing system. For example, process data, spectral data, and context data acquired about substrates being processed in the manufacturing system are inaccessible to users of the manufacturing system (e.g., operators). In some embodiments, all data stored in the data store 140 can be made inaccessible to users of the manufacturing system. In other or similar embodiments, some of the data stored in the data store 140 can be made inaccessible to users, while other parts of the data stored in the data store 140 can be made accessible to users. In some embodiments, one or more parts of the data stored in the data store 140 can be encrypted using an encryption mechanism unknown to the user (e.g., the data is encrypted using a private encryption key). In other or similar embodiments, the data store 140 can include multiple data stores, where data inaccessible to users is stored in one or more first data stores, and data accessible to users is stored in one or more second data stores.

[0078] In some embodiments, the data store 140 may be configured to store data associated with known failure patterns. A failure pattern may be one or more values ​​(e.g., vectors, scalars, etc.) associated with one or more problems or failures associated with a process chamber subsystem. In some embodiments, a failure pattern may be associated with a corrective action. For example, a failure pattern may include a parameter adjustment step to correct the problem or failure indicated by the failure pattern. For example, a predictive system or corrective action module may compare a determined failure pattern (determined from data acquired from one or more sensors in a sensor cluster) with a library of known failure patterns to determine the type of failure experienced by the subsystem, the cause of the failure, recommended corrective actions to correct the failure, and so on.

[0079] The client device 110, the manufacturing equipment 124, the sensor 126, the prediction system 160, and the data store 140 can be connected to each other via the network 130. In some embodiments, the network 130 is a public network that provides the client device 110 with access to the prediction system 160, the data store 140, the manufacturing equipment 124 (not shown), and other publicly available computing devices. In some embodiments, the network 130 is a private network that provides the client device 110 with access to the manufacturing equipment 124, the data store 140, the prediction system 160, and other privately available computing devices. The network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long-Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0080] In this embodiment, “User” can represent a single individual. However, other embodiments of this disclosure include the case where “User” is an entity controlled by multiple users and / or automated sources. For example, a collection of individual users united as a group of administrators can be considered a “User.”

[0081] Figure 2 is a schematic top view of an exemplary manufacturing system 200 according to an aspect of the present disclosure. The manufacturing system 200 can perform one or more processes on a substrate 202. The substrate 202 can be any appropriately rigid, fixed-dimension planar article suitable for manufacturing electronic devices or circuit components, such as a silicon-containing disk or wafer, a patterned wafer, or a glass plate.

[0082] The manufacturing system 200 may include a process tool 204 and a factory interface 206 coupled to the process tool 204. The process tool 204 may include a housing 208 having a transfer chamber 210 inside. The transfer chamber 210 may include one or more process chambers (also called processing chambers) 214, 216, 218 arranged around the transfer chamber 210 and coupled to the transfer chamber 210. The process chambers 214, 216, 218 can be coupled to the transfer chamber 210 via their respective ports, such as slit valves. The transfer chamber 210 may also include a transfer chamber robot 212 configured to transfer substrates 202 between the process chambers 214, 216, 218, a load lock 220, etc. The transfer chamber robot 212 may include one or more arms, each arm including one or more end effectors at the end of each arm. The end effectors can be configured to handle specific objects such as wafers, sensor disks, sensor tools, etc.

[0083] Process chambers 214, 216, and 218 can be adapted to perform any number of processes on the substrate 202. Each process chamber 214, 216, and 218 can perform the same or different substrate processes. Substrate processes can include atomic layer deposition (ALD), physical vapor deposition (PVD), chemical vapor deposition (CVD), etching, annealing, curing, pre-cleaning, and metal or metal oxide removal. Other processes can be performed on the substrate within the process chamber. Each process chamber 214, 216, and 218 can include one or more sensors configured to capture data on the substrate 202 before, after, or during the substrate process. For example, one or more sensors can be configured to capture spectral and / or non-spectral data about a portion of the substrate 202 during the substrate process. In other or similar embodiments, one or more sensors can be configured to capture data related to the environment within process chambers 214, 216, and 218 before, after, or during the substrate process. For example, one or more sensors can be configured to capture data related to the temperature, pressure, gas concentration, and other environmental factors within the process chambers 214, 216, and 218 during substrate processing.

[0084] In some embodiments, the measuring device (not shown) may be located within the process tool. In other embodiments, the measuring device (not shown) may be located within one or more process chambers 214, 216, 218. In some embodiments, a transfer chamber robot 212 may be used to position the substrate onto the measuring device. In other embodiments, the measuring device may be part of a substrate support assembly (not shown). The measuring device can provide measurement data associated with the substrate processed by the manufacturing apparatus 124. The measurement data may include values ​​such as film property data (e.g., wafer space film properties), dimensions (e.g., thickness, height, etc.), dielectric constant, dopant concentration, density, and defects. In some embodiments, the measurement data may further include values ​​of one or more surface profile property data (e.g., etching rate, etching rate uniformity, limit dimensions of one or more features contained on the surface of the substrate, limit dimensional uniformity across the surface of the substrate, edge placement error, etc.). The measurement data may relate to finished or semi-finished products. The measurement data may differ from substrate to substrate. The measurement data can be generated using, for example, reflectivity measurement techniques, polarization analysis techniques, TEM techniques, etc.

[0085] The load lock 220 can also be coupled to the housing 208 and the transfer chamber 210. The load lock 220 can be configured to interface with and couple to one side of the transfer chamber 210 and to the factory interface 206. In some embodiments, the load lock 220 can have an environmentally controlled atmosphere that can be changed from a vacuum environment (to which the substrate can be transferred to and from the transfer chamber 210) to an atmospheric pressure or near-atmospheric gas environment (to which the substrate can be transferred to and from the factory interface 206). The factory interface 206 can be any suitable enclosure, such as an equipment front-end module (EFEM). The factory interface 206 can be configured to receive substrates 202 from a substrate carrier 222 (e.g., a forward-opening unified pod (FOUP)) docked to various load ports 224 of the factory interface 206. A factory interface robot 226 (shown by a dotted line) can be configured to transfer the substrates 202 between the carrier (also called a container) 222 and the load lock 220. The carrier 222 can be a circuit board storage carrier or a replacement parts storage carrier.

[0086] The manufacturing system 200 may also be connected to a client device (e.g., client device 110, not shown) configured to provide information about the manufacturing system 200 to a user (e.g., an operator). In some embodiments, the client device may provide information to the user of the manufacturing system 200 via one or more graphical user interfaces (GUIs). For example, the client device may provide information via the GUI about the target thickness profile of the film to be deposited on the surface of the substrate 202 during the deposition process performed in process chambers 214, 216, and 218. The client device may also provide information about anomaly detection and failure classification according to embodiments described herein.

[0087] The manufacturing system 200 may also include a system controller 228. The system controller 228 may be and / or include a computing device such as a personal computer, a server computer, a programmable logic control unit (PLC), or a microcontroller. The system controller 228 may include one or more general-purpose processing units, such as a microprocessor or a central processing unit. More specifically, the processing units may be a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor or combination of instruction sets that implements other instruction sets. The processing units may also be one or more dedicated processing units such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. The system controller 228 may include data storage devices (e.g., one or more disk drives and / or solid drives), main memory, static memory, a network interface, and / or other components. The system controller 228 may execute instructions to perform one or more of the methods and / or embodiments described herein. In some embodiments, the system controller 228 can execute instructions to perform one or more operations in the manufacturing system 200 according to a process policy. Instructions can be stored in a computer-readable storage medium, which may include main memory, static memory, secondary storage, and / or processing units (while the instructions are being executed).

[0088] The system controller 228 can receive data from sensors (e.g., sensor 126, not shown) located on or within various parts of the manufacturing system 200 (e.g., processing chambers 214, 216, 218, transfer chamber 210, load lock 220, etc.). In some embodiments, the data received by the system controller 228 may include spectral and / or non-spectral data for a portion of the substrate 202. In other or similar embodiments, the data received by the system controller 228 may include data associated with the processing of the substrate 202 in processing chambers 214, 216, 218, as described above. For the purposes of this description, the system controller 228 is described as receiving data from sensors located within the process chambers 214, 216, 218. However, the system controller 228 can receive data from any part of the manufacturing system 200, and the data received from that part may be used according to the embodiments described herein. In an exemplary example, the system controller 228 can receive data from one or more sensors in the process chambers 214, 216, and 218 before, after, or during the substrate process in the process chambers 214, 216, and 218. Data received from sensors in various parts of the manufacturing system 200 can be stored in the data store 250. The data store 250 can be included as a component within the system controller 228 or as a separate component from the system controller 228. In some embodiments, the data store 250 can be the data store 140 described with respect to Figure 1.

[0089] Figure 3 shows an exemplary prediction architecture 300 according to an aspect of the present disclosure. In some embodiments, the prediction architecture 300 includes a prediction system 160, a network 130, and a data store 310 (which may be the same as the data store 140). In some embodiments, the prediction system 160 may use a model (e.g., model 190) to group two or more sensors, for example, based on sensor statistical data. For example, model 190 may receive sensor statistical data as input and generate sensor cluster data as output. In some embodiments, the prediction system 160 may include a prediction server 112, server machines 170 and 180, and a prediction server 195. Prediction server 160, server machine 170, server machine 180, and prediction server 195 may each include one or more computing devices such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, graphics processing units (GPUs), and application-specific integrated circuits (ASICs) (e.g., tensor processing units (TPUs)).

[0090] The server machine 170 includes a training set generator 172 that can generate training datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing a machine learning model 190. The machine learning model 190 can be any algorithmic model capable of learning from data. In some embodiments, the machine learning model 190 can be a predictive model. In some embodiments, the dataset generator 172 can divide the training data into training sets, validation sets, and test sets, which can be stored in the training data store 310 as part of the training statistics 312. The training statistics 312 can be accessed from the computing device prediction system 160 directly or via the network 130. In some embodiments, the prediction system 160 generates multiple training datasets.

[0091] The server machine 180 may include a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. An engine can refer to hardware (e.g., circuits, custom logic, programmable logic, microcode, processing units, etc.), software (e.g., processing units, general-purpose computer systems, or instructions executed on a custom machine), firmware, microcode, or a combination thereof. The training engine 182 can train one or more machine learning models 190. A machine learning model 190 can refer to a model artifact created by the training engine 182 using training data (also referred to herein as the training set) which includes training inputs and corresponding target outputs (the correct answers for each training input). The training engine 182 can find patterns in the training data that map training inputs to target outputs (predicted answers) and provide machine learning models 190 that capture these patterns. Machine learning models 190 can use one or more of the following: statistical modeling, support vector machines (SVM), radial basis functions (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithm (k-NN), linear regression, random forest, and neural networks (e.g., artificial neural networks).

[0092] One type of machine learning model that can be used to perform some or all of the above tasks is an artificial neural network, such as a deep neural network. An artificial neural network generally contains feature representation components that have classifier or regression layers that map features to a desired output space. A convolutional neural network (CNN), for example, hosts multiple layers of convolutional filters. Pooling is performed, and nonlinearity can be dealt with in lower layers, and a multilayer perceptron is generally added on top of those lower layers to map the top layer features extracted by the convolutional layers to a decision (e.g., classification output). Deep learning is a class of machine learning algorithms that use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. Deep neural networks can be trained in supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) forms. A deep neural network contains a hierarchy of layers, where different layers learn different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and complex representation. In plasma process tuning, for example, the raw input could be a process result profile (e.g., a thickness profile showing one or more thickness values ​​across the entire surface of the substrate), a second layer could consist of feature data associated with the state of one or more zones of the control elements of the plasma process system (e.g., zone orientation, plasma exposure time, etc.), and a third layer could include a starting policy (e.g., a policy used as a starting point to determine an updated process policy for processing the substrate and producing process results that meet threshold criteria). Notably, the deep learning process can learn on its own which features are best placed at which levels. The "deep" in "deep learning" refers to the number of layers to which the data is transformed. More precisely, a deep learning system has a considerable depth of confidence assignment paths (CAPs). A CAP is a chain of transformations from input to output.A CAP describes the latent causal relationship between an input and an output. In the case of a feedforward neural network, the depth of the CAP can be the depth of the network, which can be the number of hidden layers + 1. In the case of a recurrent neural network, where a signal can propagate through layers two or more times, the depth of the CAP is potentially unlimited.

[0093] In one embodiment, one or more machine learning models are recurrent neural networks (RNNs). An RNN is a type of neural network that includes memory, allowing the neural network to capture temporal dependencies. An RNN can learn input-output mappings that depend on both current and past inputs. An RNN can deal with past and future flow measurements and make predictions based on this continuous measurement information. An RNN can be trained using a training dataset and can produce a fixed number of outputs (e.g., determining a set of substrate processing rates, determining modifications to a substrate process policy). One type of RNN that can be used is a long short-term memory (LSTM) neural network.

[0094] Training a neural network can be achieved in a supervised learning manner, which involves feeding the network a training dataset consisting of labeled inputs, observing its output, defining the error (by measuring the difference between the output and the labeled value), and adjusting the network's weights across all layers and nodes of the network to minimize the error, using techniques such as deep gradient descent and backpropagation. In many applications, repeating this process across many labeled inputs in the training dataset results in a network that can produce the correct output even when presented with inputs different from those present in the training dataset.

[0095] A training dataset can be formed using a training dataset that includes hundreds, thousands, tens of thousands, hundreds of thousands, or more sensor data and / or process result data (e.g., measurement data such as one or more thickness profiles associated with sensor data).

[0096] To achieve training, the processing logic can input the training dataset into one or more untrained machine learning models. The machine learning models can be initialized before inputting the first input into them. The processing logic trains the untrained machine learning models based on the training dataset to produce one or more trained machine learning models that perform the various operations described above. Training can be performed by inputting one or more of the sensor data into the machine learning models one at a time.

[0097] A machine learning model processes inputs and produces outputs. An artificial neural network includes an input layer consisting of data point values. The next layer is called a hidden layer, and each node in the hidden layer receives one or more input values. Each node contains parameters (e.g., weights) that are applied to the input values. Thus, each node essentially inputs the input values ​​into a multivariate function (e.g., a nonlinear mathematical transformation) to produce output values. The next layer can be another hidden layer or an output layer. In either case, the nodes in the next layer receive output values ​​from the nodes of the previous layer, and each node applies weights to these values ​​and then produces its own output values. This can happen in each layer. The final layer is the output layer, and there is one node for each class, prediction, and / or output that the machine learning model can produce.

[0098] Therefore, the output may include one or more predictions or inferences. In some embodiments, the output predictions or inferences may include one or more predictions such as classification of sensor groups or ranking of sensors. In some embodiments, the output predictions or inferences may include one or more predictions such as anomaly data, failure data, or failure detection limits. The processing logic determines an error (i.e., classification error) based on the difference between the output of the machine learning model (e.g., predictions or inferences) and the target labels associated with the input training data. Based on the error, the processing logic adjusts the weights of one or more nodes of the machine learning model. An error term or delta may be determined for each node of the artificial neural network. Based on this error, the artificial neural network adjusts one or more of its parameters (weights of one or more inputs of the node) of one or more of its nodes. Parameters may be updated in a backpropagation manner, with the top layer nodes being updated first, followed by the nodes of the next layer, and so on. The artificial neural network consists of multiple layers of "neurons," each layer receiving values ​​from the neurons of the previous layer as input. The parameters of each neuron include weights associated with the values ​​received from each of the neurons of the previous layer. Therefore, tuning parameters can include adjusting the weights assigned to each of the inputs of one or more neurons in one or more layers within an artificial neural network.

[0099] After one or more training rounds, the processing logic can determine whether the stopping criteria have been met. The stopping criteria can be a target level of accuracy, a target number of processed images from the training dataset, a target change in a parameter across one or more previous data points, a combination of these, and / or other criteria. In one embodiment, the stopping criteria are met when at least a minimum number of data points have been processed and at least a threshold accuracy has been achieved. The threshold accuracy can be, for example, 70%, 80%, or 90% accuracy. In one embodiment, the stopping criteria are met when the accuracy improvement of the machine learning model has stopped. If the stopping criteria are not met, further training is performed. If the stopping criteria are met, training can be completed. Once the machine learning model is trained, the model can be tested using a reserved portion of the training dataset.

[0100] Once one or more trained machine learning models 190 are generated, these models may be stored in the prediction server 195 as prediction components 197, or as components of prediction components 197.

[0101] The validation engine 184 may be able to validate the machine learning model 190 using the corresponding set of features from the validation set generated by the training set generator 172. Once the model parameters have been optimized, model validation may be performed to determine whether the model has improved and to determine the current accuracy of the deep learning model. The validation engine 184 can determine the accuracy of the machine learning model 190 based on the corresponding set of features from the validation set. The validation engine 184 may discard trained machine learning models 190 that have an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be able to select trained machine learning models 190 that have an accuracy that meets a threshold accuracy. In some embodiments, the selection engine 185 may be able to select the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190.

[0102] The test engine 186 may be able to test the trained machine learning model 190 using the corresponding set of features from the test set of the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features from the training set can be tested using a first set of features from the test set. Based on the test set, the test engine 186 can determine which trained machine learning model 190 has the highest accuracy among all the trained machine learning models.

[0103] As will be described in detail below, the prediction server 195 includes a prediction component 197 that provides data indicating the grouping or ranking of sensors and runs a trained machine learning model 190 on input sensor data, statistical data, and other data items to obtain one or more outputs. The prediction server 195 can further provide fault detection data and / or anomaly detection data, which will be described in more detail below.

[0104] It should be noted that in some other implementations, the functionality of server machines 170 and 180, as well as the prediction server 195, may be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 can be integrated into a single machine, while in some other or similar embodiments, server machines 170 and 180, as well as the prediction server 195, can be integrated into a single machine.

[0105] In general, functions described in one embodiment as being performed by server machine 170, server machine 180, and / or prediction server 195 can also be performed on client device 110. In addition, functions that belong to a particular component can be performed by different components or multiple components working together.

[0106] In some embodiments, the manufacturing system may include two or more process chambers. For example, the exemplary manufacturing system 200 in Figure 2 shows multiple process chambers 214, 216, and 218. Note that in some embodiments, the data acquired to train the machine learning model 190 and the data collected to be provided as input to the machine learning model may be associated with the same process chamber of the manufacturing system. In other or similar embodiments, the data acquired to train the machine learning model and the data collected to be provided as input to the machine learning model may be associated with different process chambers of the manufacturing system. In other or similar embodiments, the data acquired to train the machine learning model may be associated with a process chamber in a first manufacturing system, and the data collected to be provided as input to the machine learning model may be associated with a process chamber in a second manufacturing system.

[0107] Figure 9 is a flowchart of a method 900 for generating sensor group data according to an aspect of the present disclosure. Method 900 is performed by processing logic that may include hardware (circuits, dedicated logic, etc.), software (such as that which runs on a general-purpose computer system or a dedicated machine), firmware, or any combination thereof. In one embodiment, Method 900 can be performed by a computer system such as the computer system architecture 100 of Figure 1. In other or similar embodiments, one or more operations of Method 900 can be performed by one or more other machines not shown in the figures. In some embodiments, one or more operations of Method 900 can be performed by a client device 110, a server machine 170, a server machine 190, and / or a prediction server 195. In some embodiments, Method 900 can be performed by a grouping module 113 which can classify sensors into one or more specific groups using a distribution algorithm.

[0108] In operation 910, the processing logic acquires output data related to the sensor. The sensor can be associated with one or more process chambers of the manufacturing apparatus 124, or one or more process chambers of multiple manufacturing apparatuses 124, and the data can be acquired from, for example, the manufacturing apparatus 124, the data store 140, the client device 110, etc.

[0109] In operation 920, the processing logic generates a first distribution based on the data and time. This first distribution may include a series of data points (e.g., output values) associated with the sensor over time. In some embodiments, the distribution may include a Gaussian distribution (determining the mean and one or more deviations from the mean), a graph, and the like.

[0110] In operation 930, the processing logic generates a second distribution based on the data and the tool lifetime. Each distribution may include a series of data points (e.g., output values) over the tool lifetime. In some embodiments, the distribution may include a Gaussian distribution, a graph, and so on.

[0111] In operation 940, the processing logic generates coefficients of variation based on both distributions. For example, the processing logic can calculate the coefficient of variation for each correlated sensor value.

[0112] In operation 950, the processing logic generates a correlation coefficient based on both distributions. For example, the processing logic can calculate the correlation coefficient for each correlated sensor value. In one example, the correlation coefficient is a number between -1 and 1.

[0113] In operation 960, the processing logic determines whether the coefficient of variation (CV) is below a threshold. The threshold can be determined using experiments, machine learning, user input, etc. If the processing logic determines that the coefficient of variation is below the threshold, it proceeds to operation 970A. If the processing logic determines that the coefficient of variation is above the threshold, it proceeds to operation 970B.

[0114] In operations 970A to B, the processing logic determines whether the correlation coefficient (CC) of one or more distributions meets a threshold criterion. For example, the processing logic can determine whether the correlation coefficient has a predetermined value for a value of 1. In operation 970A, in response to the correlation coefficient meeting the threshold criterion, the processing logic classifies the sensor as a setpoint sensor in operation 980. In response to the correlation coefficient not meeting the threshold criterion, the processing logic classifies the sensor as a variability sensor in operation 995.

[0115] In operation 970B, if the correlation coefficient of the tool life distribution meets the threshold criterion, the processing logic classifies the sensor as a tool life sensor in operation 990. If the correlation coefficient does not meet the threshold criterion, the processing logic classifies the sensor as a variability sensor in operation 995. Note that the thresholds associated with operations 970A and 970 may be the same or different.

[0116] Figure 10 is a flowchart of a fault detection method 1000 based on aggregate statistics, according to an aspect of this disclosure. Method 1000 is performed by processing logic that may include hardware (circuits, dedicated logic, etc.), software (such as that which runs on a general-purpose computer system or a dedicated machine), firmware, or any combination thereof. In one embodiment, Method 1000 can be performed by a computer system such as the computer system architecture 100 in Figure 1. In other or similar embodiments, one or more operations of Method 1000 can be performed by one or more other machines not shown in the figure. In some embodiments, one or more operations of Method 1000 can be performed by a client device 110, a server machine 170, a server machine 180, and / or a prediction server 195.

[0117] In operation 1010, the processing logic obtains raw sensor statistics from multiple process chambers for sensors (e.g., sensor 126) that collect data during the processing operation (e.g., process execution). The set of sensors that are activated (or collect data from) during the processing operation can be selected by the processing unit based on the specifications of the processing operation. The raw sensor statistics can characterize multiple measurements associated with the activated (sampled) sensors. Statistics describing the measurements collected by each or some of the sensors may include various parameters such as median, mode, variance, standard deviation, range, maximum, minimum, skewness, or kurtosis.

[0118] In operation 1020, the processing logic aggregates the sensor data into a single dataset. For example, the processing logic can combine sensor data from each process chamber into a single dataset.

[0119] In operation 1030, the processing logic generates a distribution for the dataset. In some embodiments, the processing logic generates a Gaussian distribution.

[0120] In operation 1040, the processing logic identifies one or more fault detection limits. In one embodiment, the fault detection limit may include a standard deviation threshold of a Gaussian distribution.

[0121] In operation 1050, the processing logic generates a fault detection management chart. The fault detection management chart can be used to monitor faults during subsequent process execution. In some embodiments, after maintenance of one or more process chambers is completed, the processing logic can acquire sensor data and plot the sensor data on the fault detection management chart in real time or near real time. In response to the sensor output value crossing the fault detection limit, the processing logic can display a warning, take corrective action, etc. In some embodiments, the processing logic can determine a control line associated with the sensor data and determine the slope of the control line. In response to the slope value meeting a threshold criterion (e.g., crossing the fault detection limit), the processing logic can display a warning, take corrective action, etc. In some embodiments, the processing logic can determine the y-intercept of the control line. In response to the y-intercept meeting a threshold criterion (e.g., crossing the fault detection limit), the processing logic can display a warning, take corrective action, etc.

[0122] Figure 11 is a flowchart of Method 1100 for determining projected control lines according to an aspect of the present disclosure. Method 1100 is performed by processing logic that may include hardware (circuits, dedicated logic, etc.), software (such as that which runs on a general-purpose computer system or a dedicated machine), firmware, or any combination thereof. In one embodiment, Method 1100 can be performed by a computer system such as the computer system architecture 100 in Figure 1. In other or similar embodiments, one or more operations of Method 1100 can be performed by one or more other machines not shown in the figures. In some embodiments, one or more operations of Method 1100 can be performed by a client device 110, a server machine 170, a server machine 180, and / or a prediction server 195.

[0123] In operation 1110, the processing logic generates a fault detection management chart. For example, the processing logic can perform one or more operations shown in Figure 9 to generate the fault detection management chart.

[0124] In operation 1120, the processing logic plots a series of sensor values ​​on a fault detection management chart. The sensor values ​​can be associated with one or more current process executions running within the process chamber. The sensor values ​​can be associated with one or more sensors.

[0125] In operation 1130, the processing logic generates a control line based on the sensor values. In some embodiments, the control line can be generated by determining the best-fit line for the sensor data, using any other method that can determine the relationships between the sensor data, or by using the prediction server 195.

[0126] In operation 1140, the processing logic determines the slope value of the control line. In some embodiments, the slope can be determined using the equation y = m*x + b.

[0127] In operation 1150, the processing logic generates projected control lines. In some embodiments, the processing logic can generate projected control lines by extending the control lines using slope values. The projected control lines can be extended over a predetermined period of time. The projected control lines can be regraphed at predetermined intervals (for example, every five current data points are plotted, every time a current data point is plotted, etc.).

[0128] In operation 1160, the processing logic determines whether the projected fault line meets a threshold criterion (e.g., crosses the fault detection limit). In response to the projected control line crossing the fault detection limit, the processing logic proceeds to operation 1170 to generate a warning. Otherwise, the processing logic proceeds to operation 1120 to obtain additional sensor values ​​and generate a new projected control line (in operation 1150).

[0129] In some embodiments, the processing logic can set triggers based on the slope value of the control line, as described above in relation to Figure 8B. In response to the slope value meeting a threshold criterion (for example, the slope value of the control line exceeding or falling below a predetermined value), the processing logic can generate a warning.

[0130] In some embodiments, the processing logic can determine the slope of sensor data. The slope can be used to determine whether one or more sensors, process chambers, process chamber subsystems, etc., are experiencing a failure. In some embodiments, the processing logic can compare the slope to one or more fault detection limits to determine whether a failure exists. For example, the processing logic can determine the slope of a sensor output value and compare the sensor slope to the slope of one or more fault detection limits. In response to the difference between the sensor slope value and the fault detection limit meeting a threshold criterion (e.g., the sensor slope value is greater or less than the fault detection limit slope by a predetermined value, percentage, etc.), the processing logic can issue a warning and take corrective action.

[0131] In some embodiments, the processing logic can update the fault detection limit in response to determining that a value (e.g., the number of boards processed in the process chamber since the last maintenance operation on the processing chamber, or the elapsed time) meets a threshold criterion. In particular, the processing logic can track the duration of a production run. In response to determining that the duration of a production run meets a threshold criterion (e.g., the number of boards processed meets the threshold, or the elapsed time since the start of the production run meets the threshold), the processing logic can update the value of the fault detection limit (e.g., reset or set to a new value). In some embodiments, the threshold criterion can reflect desired or scheduled maintenance operations. In one example, the fault detection limit can be reset to a value set at the start of the production run. In another example, the fault detection limit can be updated to a new value determined by the fault detection module 115 or the chart generator 116.

[0132] Figure 12 is a block diagram showing a computer system 1200 according to a particular embodiment. In some embodiments, the computer system 1200 can connect to other computer systems (for example, via a network such as a local area network (LAN), intranet, extranet, or the Internet). The computer system 1200 can operate as a server or client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 1200 may be provided by a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) specifying actions to be taken by such device. Furthermore, the term “computer” includes any set of computers that individually or collectively execute a set of instructions (or sets of instructions) to perform one or more of the methods described herein.

[0133] In a further embodiment, the computer system 1200 may include a processing unit 1202, a volatile memory 1204 (e.g., random access memory (RAM)), a non-volatile memory 1206 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 1216, which can communicate with each other via a bus 1208.

[0134] The processing unit 1202 may be provided by one or more processors, such as a general-purpose processor (e.g., a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of multiple types of instruction sets) or a dedicated processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0135] The computer system 1200 may further include a network interface device 1222 (for example, coupled to network 1274). The computer system 1200 may also include a video display unit 1210 (for example, an LCD), a character / number input device 1212 (for example, a keyboard), a cursor control device 1214 (for example, a mouse), and a signal generation device 1220.

[0136] In some embodiments, the data storage device 1216 may include a non-temporary computer-readable storage medium 1224 which may store instructions 1226 that code one or more of the methods or functions described herein, including instructions that code components of Figure 1 (e.g., SCM 111, SSM 112, grouping module 113, ADM 114, FDM 115, chart generator 116, and corrective action module 116), and instructions for performing the methods described herein.

[0137] Instruction 1226 may also reside entirely or partially in the volatile memory 1204 and / or the processing unit 1202 during its execution by the computer system 1200, and thus the volatile memory 1204 and the processing unit 1202 may also constitute a machine-readable storage medium.

[0138] Although the computer-readable storage medium 1224 is shown as a single medium in the illustrative examples, the term “computer-readable storage medium” includes a single or multiple mediums (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of executable instructions. The term “computer-readable storage medium” also includes any tangible medium capable of storing or encoding a set of instructions for execution by a computer, which causes a computer to execute one or more of the methods described herein. The term “computer-readable storage medium” includes, but is not limited to, solid memory, optical media, and magnetic media.

[0139] The methods, components, and features described herein may be implemented by individual hardware components or integrated into the functionality of other hardware components such as ASICs, FPGAs, DSPs, or similar devices. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within hardware devices. Moreover, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in computer programs.

[0140] Unless otherwise specified, terms such as “receive,” “execute,” “provide,” “acquire,” “cause,” “access,” “determine,” “add,” “use,” and “train” refer to actions and processes performed or implemented by a computer system that manipulate data represented as physical (electronic) quantities in the registers and memory of the computer system and convert them into other data similarly represented as physical quantities in the memory or registers of the computer system, or other such information storage devices, transmission or display devices. Furthermore, terms such as “first,” “second,” “third,” and “fourth” as used herein are labels to distinguish different elements and cannot have an ordering meaning due to their numerical designation.

[0141] The examples described herein also relate to apparatus for performing the methods described herein. Such apparatus may include a general-purpose computer system that is specifically constructed for performing the methods described herein, or that is selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0142] The methods and exemplary embodiments described herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used in accordance with the teachings described herein, or it may be convenient to construct more specialized devices to perform each of the methods and / or their individual functions, routines, subroutines, or operations described herein. Examples of the construction of various such systems are described above.

[0143] The above description is illustrative and not limiting. While this disclosure has been described with reference to certain exemplary examples and embodiments, it should be recognized that this disclosure is not limited to the described examples and embodiments. The scope of this disclosure should be determined with reference to the following claims, along with the entire scope of equivalents to which the claims are granted.

Claims

1. The processing unit performs the steps of acquiring current sensor data associated with the sensors of the substrate manufacturing system, The steps include determining the slope value associated with the current sensor data, In response to determining that the slope value has met the threshold criteria associated with the fault detection limit, the step of performing at least one of the following: generating a warning or taking corrective action, Methods that include...

2. The method according to claim 1, wherein the slope of the fault detection limit includes a first value for a first period of production execution and a second value for a second period of production execution.

3. The steps include determining a control line based on the current sensor data, The steps include determining the control line and the control line projected based on the slope value, The steps include displaying the projected control lines on a graphical user interface, The method according to claim 1, further comprising:

4. The steps include determining whether the y-intercept value associated with the control line satisfies a further threshold criterion associated with the fault detection limit, In response to determining that the value of the y-intercept has met a further threshold criterion associated with the fault detection limit, the step of performing at least one of the steps of generating the warning or performing the corrective action, The method according to claim 1, further comprising:

5. The method according to claim 1, wherein the slope value is updated in response to the reception of additional sensor data.

6. The steps include: acquiring multiple datasets, each containing sensor output data from each of the multiple sensors associated with a corresponding process chamber among multiple process chambers; The steps include: combining the aforementioned multiple datasets into an aggregated dataset; The steps include generating the distribution of the aggregated dataset, A step of identifying the fault detection limit based on the deviation value generated from the distribution, The method according to claim 1, further comprising:

7. The steps include: acquiring output data associated with one of the multiple sensors; A step of generating a first distribution based on the output data and a series of time values, The steps include generating a second distribution based on the output data and a series of tool life values, A step of generating a series of coefficients of variation based on the first distribution and the second distribution, A step of generating a series of correlation coefficients based on the first distribution and the second distribution, The steps include assigning the sensors to a group in response to the fact that the series of coefficients of variation satisfy a first threshold criterion and the correlation coefficient satisfies a second threshold criterion, The method according to claim 6, further comprising:

8. The method according to claim 7, wherein the group reflects sensors whose output values ​​do not drift over time.

9. In response to the fact that the series of coefficients of variation does not satisfy the first threshold criterion and the series of correlation coefficients satisfy the second threshold criterion, the step of assigning the sensor to a tool life group that reflects a sensor whose output value drifts over time. The method according to claim 7, further comprising:

10. In response to the fact that the series of correlation coefficients did not meet the second threshold criterion, the step of assigning the sensor to a variability group that reflects the sensors generating asymmetric data. The method according to claim 7, further comprising:

11. The method according to claim 7, wherein at least one of the first distribution or the second distribution is a Gaussian distribution.

12. The step of displaying health indices associated with a set of related sensors assigned to a group from multiple process chambers via a graphical user interface. The method according to claim 7, further comprising:

13. Step 1: In response to determining that the production run period meets further threshold criteria, update the value associated with the fault detection limit. The method according to claim 1, further comprising:

14. Memory devices and, A processing unit operably coupled to the memory device, A step to acquire current sensor data associated with the sensors in the manufacturing system, The steps include determining the slope value associated with the current sensor data, In response to determining that the slope value has met the threshold criteria associated with the fault detection limit, the steps include generating a warning or taking at least one of the following actions: A processing unit that performs operations including, An electronic device manufacturing system equipped with the following features.

15. The system according to claim 14, wherein the slope of the fault detection limit includes a first value for a first period of production execution and a second value for a second period of production execution.

16. The steps include determining a control line based on the current sensor data, The steps include determining the control line and the control line projected based on the slope value, The steps include displaying the projected control lines on a graphical user interface, The method according to claim 14, further comprising:

17. The steps include determining whether the y-intercept value associated with the control line satisfies a further threshold criterion associated with the fault detection limit, The steps include: in response to determining that the value of the y-intercept has met a further threshold criterion associated with the fault detection limit, generating the warning or performing the corrective action; The system according to claim 14, further comprising:

18. The system according to claim 14, wherein the slope value is updated in response to the reception of additional sensor data.

19. The aforementioned operation, The steps include: acquiring multiple datasets, each containing sensor output data from each of the multiple sensors associated with a corresponding process chamber among multiple process chambers; The steps include: combining the aforementioned multiple datasets into an aggregated dataset; The steps include generating the distribution of the aggregated dataset, A step of identifying the fault detection limit based on the deviation value generated from the distribution, The system according to claim 14, further comprising:

20. A non-temporary computer-readable storage medium, which, when executed by a processing unit operably coupled to memory, The steps include: acquiring multiple datasets, each containing sensor output data from each of the multiple sensors associated with a corresponding process chamber among multiple process chambers; The steps include: combining the aforementioned multiple datasets into an aggregated dataset; The steps include generating the distribution of the aggregated dataset, A step of identifying a fault detection limit based on the deviation value generated from the distribution, A non-temporary computer-readable storage medium containing instructions for performing operations including [a specific operation].

21. The non-temporary computer-readable storage medium according to claim 20, wherein the slope of the fault detection limit includes a first value for a first period of production execution and a second value for a second period of production execution.

Images