Minimally supervised learning for determining causes of outlying data points

Abstract

A system can identify an anomalous output among a plurality of outputs during a process step. The system can receive manufacturing attributes associated with each of the plurality of outputs containing the anomalous output. The system can use a machine learning model to construct at least one isolation tree model, including a plurality of parent nodes each corresponding to a splitting condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly caused the anomalous output.

Description

[Technical Field]

[0001] The disclosed technology generally relates to semiconductor manufacturing and measurement methods and equipment. More specifically, the disclosed technology relates to techniques and systems for identifying the causes of abnormal outputs in semiconductor manufacturing processes. [Previous Technology]

[0002] In semiconductor manufacturing, complex structures can be fabricated using a sequence of thin film deposition, photolithography, and etching. Photolithography can be used to define areas to be exposed to a photo-patterned mask layer (e.g., a photoresist layer or a hard mask layer), remove portions of the mask layer to define a stencil pattern with small feature sizes, apply an etchant to remove portions of the underlying layer using the stencil pattern, and repeat these steps multiple times for various layers of a device to fabricate complex structures. Various tools, including processing tools and metrology tools, can be used in semiconductor manufacturing.

[0003] Due to the complex nature of semiconductor manufacturing and the numerous steps involved, process anomalies may be difficult to detect online or isolate the steps(s) associated with them. Furthermore, due to cost considerations, only a sample of the wafer is inspected within a subset of the process steps. Therefore, when an anomalous output is detected in a process step or in a final product, it may be difficult to determine the cause of the step associated with the anomalous output from the semiconductor process (e.g., a semiconductor device having one or more features that do not meet specifications) or to identify that step. This can cause delays in semiconductor manufacturing when identifying and correcting the cause of the anomalous output. [Summary of the Invention]

[0004] For the purpose of summarizing the present invention and the advantages achieved relative to the prior art, certain objects and advantages of the present invention are described herein. Not all such objects and advantages may be achieved in any particular embodiment. Thus, for example, those skilled in the art will recognize that the present invention may be embodied or practiced in a manner that achieves or optimizes one advantage or group of advantages (as taught herein) without necessarily achieving other objects or advantages (as taught or implied herein).

[0005] All such embodiments are intended to be within the scope of the invention disclosed herein. Those skilled in the art will readily understand from the following detailed description of preferred embodiments with reference to the accompanying drawings that these and other embodiments are not limited to any particular preferred embodiment disclosed.

[0006] In one embodiment, a method for associating an anomalous output of a semiconductor process with a manufacturing attribute includes: identifying the anomalous output among a plurality of outputs in a process step of the process; receiving a manufacturing attribute associated with each of the plurality of outputs containing the anomalous output; and using a machine learning model to construct at least one isolation tree model, which includes a plurality of parent nodes each corresponding to a splitting condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output. Each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output.

[0007] In another embodiment, a non-transitory computer-readable storage medium stores instructions that, when executed by a system of one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor manufacturing process; receive manufacturing attributes associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, which includes a plurality of parent nodes each corresponding to a partitioning condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output. Each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output.

[0008] In another embodiment, a system for associating an anomalous output of a semiconductor process with a manufacturing attribute of the process includes one or more processors and a non-transitory computer-readable storage medium storing instructions that, when executed by the one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor process; receive a manufacturing attribute associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, which includes a plurality of parent nodes each corresponding to a partitioning condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output. Each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output.

Implementation Method

[0015] Cross-reference to related applications: Any and all applications whose foreign or domestic priority claims are identified in the application data sheet filed together with this application, pursuant to 37 CFR 1.57.

[0016] This application claims priority to Provisional Patent Application No. 63 / 685178, filed on August 20, 2024, entitled “MINIMALLY SUPERVISED LEARNING FOR DETERMINING CAUSES OF OUTLYING DATA POINTS”, the contents of which are incorporated herein by reference in their entirety.

[0017] The following detailed description of certain embodiments presents various descriptions of specific embodiments. However, the innovations described herein may be embodied in, for example, as defined and covered by the embodiments. In this description, reference is made to the drawings, wherein the same element symbols may indicate the same or functionally similar elements. It should be understood that the elements drawn in the figures are not necessarily drawn to scale. Furthermore, it should be understood that some embodiments may include more elements and / or a subset of the elements drawn than are shown in a single drawing. In addition, some embodiments may incorporate any suitable combination of features from two or more drawings.

[0018] FIG1 is a high-level schematic diagram of an analysis and computing system according to various forms of the present invention for analyzing a semiconductor manufacturing process. As shown, system 100 includes a manufacturing system 102, an analysis and computing system 112 and a measurement system 114.

[0019] In some embodiments, manufacturing system 102 may represent a collection of any system or subsystem capable of performing at least a portion of a process (such as a semiconductor process). Manufacturing system 102 includes one or more manufacturing apparatuses 108 performing physical steps of the process, and a control system 110 providing control inputs to the manufacturing apparatuses 108. In a semiconductor process, some examples of manufacturing apparatuses 108 may include (but are not limited to) (e.g.) a thin film deposition apparatus, a photolithography apparatus, an etching apparatus, a stack alignment apparatus, a chemical mechanical planarization apparatus, an annealing apparatus, and a cleaning apparatus. Some examples of semiconductor process steps performed by such apparatus include (but are not limited to) thin film deposition, photolithography, etching, stack alignment, annealing, cleaning, and chemical mechanical planarization.

[0020] During operation of the manufacturing apparatus 108, one or more external sensors 104 and / or one or more trajectory sensors 106 generate data that can be transmitted to and consumed by the analysis and computing system 112. The trajectory sensor 106 may be mounted as part of a manufacturing apparatus for, for example, in-situ monitoring or measurement. In some embodiments, the trajectory sensor 106 performing in-situ monitoring or measurement may include one or more sensors that measure characteristics of the manufacturing apparatus 108 or an action performed by the manufacturing apparatus 108. Examples of characteristics measured by the trajectory sensor 106 in these embodiments include (but are not limited to) one or more of the following: heating element region or wafer temperature; mass flow rate of inlet and / or outlet gas or liquid streams; chamber pressure; power supply current, voltage, power, and / or frequency; lithography parameters; pad pressure and rotation parameters for the optical emission spectral wavelength bands used in chemical mechanical polishing or waste streams.

[0021] In some embodiments, the external sensor 104 may include one or more sensors that measure characteristics of the environment in which the manufacturing apparatus 108 operates, such characteristics that may affect the conditions of an output of the manufacturing apparatus 108 for one or more reasons. Examples of characteristics that can be measured by the external sensor 104 include (but are not limited to) a timestamp of an action taken by a manufacturing apparatus 108, an ambient temperature, or a relative humidity, or more thereof. In some embodiments, prior values ​​may also be collected and reported by the external sensor 104 and / or the trajectory sensor 106. Examples of prior values ​​may include (but are not limited to) a wafer number, a chamber accumulator counter value, a hot plate identifier, and one or more measurements from a previous process step.

[0022] Once the manufacturing apparatus 108 performs one or more steps on an input (e.g., a wafer), the metrology system 114 can measure one of the outputs of the manufacturing apparatus 108 (e.g., an output wafer) to analyze the accuracy of the operations performed by the manufacturing apparatus 108. One or more metrology sensors may be mounted as part of a metrology device for, for example, non-in-situ monitoring or measurement. In some other embodiments, the metrology sensors performing non-in-situ monitoring or measurement may include one or more sensors that measure the physical, electrical, or optical characteristics of the result of a procedure performed by the manufacturing apparatus 108 or an action performed by the manufacturing apparatus 108. Examples of characteristics measured by the metrology sensors in these embodiments include (but are not limited to) one or more of the following: thickness, film uniformity (e.g., thickness uniformity within a wafer), film stress (e.g., wafer bending), feature dimensions (e.g., feature width), feature morphology (e.g., feature angle), optical parameters (e.g., refractive index), and defect profiles (e.g., particles). The metrology system 114 can generate one or more measurement values ​​based on its output, including (but not limited to) one or more of a thickness, a stress, a refractive index, a sidewall angle, and an etching critical dimension. The measurement values ​​and / or values ​​from the sensors can then be provided to the analysis and processing system 112.

[0023] In semiconductor manufacturing processes such as those illustrated in Figure 1, a persistent problem is determining why a particular manufacturing output (such as a wafer) is abnormal (e.g., not conforming to one or more specifications), or at least isolating a manufacturing step associated with the abnormality. Multiple adjustable settings exist on the manufacturing apparatus 108, and the output of the manufacturing apparatus 108 is typically affected by both the state of the manufacturing apparatus 108 and various external factors, such as those reported by the trajectory sensor 106, external sensor 104, and metering system 114. That is, because there can be a large number of variables reported by the sensors and a large number of variables reported by the metering system 114, determining the cause of a particular abnormality among all the reported data can be computationally challenging. Therefore, there is a need for automated and computationally economical techniques capable of interpreting abnormalities in manufacturing outputs such as those of semiconductor wafers.

[0024] In some examples, one or more anomaly detection models, such as isolated forests or isolated trees, can be used to determine the cause of a particular anomaly. In these techniques, a tree can be created from all data (e.g., a set of measurements from a metrology system 114 for multiple wafers), and a leaf node containing anomalous data can be identified. A technique for analyzing the tree (Shapley superposition interpretation (SHAP) values) can be used to determine one interpretation of why anomalous data is sorted as is. However, some of these techniques may be inefficient for several reasons. For example, these techniques process all data (including anomalous and normal data) to detect anomalous data, thereby consuming very high computational resources. The present invention describes an improvement in anomaly detection by using machine learning techniques that can improve the efficiency of detecting anomalous outputs.

[0025] Figure 2 illustrates an example of a machine learning technique that can be used to detect anomalous data. In the example, a plurality of data points (e.g., a measurement set from a wafer of a metrology system 114) are organized into an isolation tree 204. For ease of discussion, a balanced binary tree is shown, but an actual isolation tree 204 may contain branches of varying lengths and / or more than two child nodes for each node. Each node in the tree partitions the data points based on a threshold value of a feature of the measurement set, and the length from the root node to a leaf node containing a single measurement set (e.g., a data point from a given wafer) is an indicator of the degree of anomalousness of the data point. In some implementations, a number of isolation trees (such as isolation tree 204) can be constructed using a random selection of the features and threshold values ​​of each node to create an isolation forest, and the average path length can be used to determine anomalous data points. In the illustrated example, nodes drawn in black contain anomalous data points.

[0026] Once outlier data points are identified by constructing an isolation tree / forest, SHAP values ​​(or other similar techniques) can be used to help determine which features contributed to the anomalous nature of the data points. While this can produce a result, the difficulty of producing a result increases exponentially with the complexity of the problem (e.g., when the number of variables considered increases). Because processes such as semiconductor manufacturing typically result in a very large number of data points and measurements / features per data point, the problem quickly becomes challenging when an isolation forest is based on an isolation tree (such as isolation tree 204).

[0027] Figure 3 illustrates an improved technique for interpreting anomalies using an isolation tree / forest according to various forms of the present invention. Since the output of a semiconductor process can be measured (e.g., using metrology system 114), the set of anomalies in the measurements or data points can be known (e.g., by detecting one or more defects in a wafer using metrology system 114). Therefore, it is not necessary to construct the entire isolation tree / forest illustrated in Figure 2 to find the anomalous data points. For example, at each level of the tree, the measurement set can be divided into child nodes as in isolation tree 204, instead of processing all child nodes, and child nodes containing anomalous data points can be processed without processing at least some of the other child nodes (e.g., all other child nodes). This produces the sparse isolation tree 304 illustrated in Figure 3. In the illustrated example, solid line nodes indicate nodes that have been determined, and nodes with X and dashed lines indicate nodes that have not been further processed. Therefore, instead of the 31 nodes shown in Figure 2, an isolation tree 204 is used, which contains only 5 nodes that are processed. The SHAP value can still be calculated based on these 5 nodes and used to help interpret outlier data points. By processing only nodes known to be outliers, the total size of the tree can be reduced by several orders of magnitude from having to build the entire tree, thus making the program easier to process and suitable for interpreting anomalies.

[0028] In some embodiments, some child nodes that do not contain anomalous data points may also be processed (e.g., some nodes marked with X and some child nodes from such nodes). In some examples, an increase in the total number of nodes considered may improve the estimated average depth of an unbuilt subtree, which may be considered in the overall analysis of the isolation tree (e.g., using SHAP values ​​or other suitable techniques). An increase in the total number of nodes considered may compromise the total computation time (e.g., increasing the processing time required to build the isolation tree). Various techniques may be used to determine which child nodes that do not contain anomalous data points are processed. In some implementations, after a split condition containing anomalous data points, only a set number of child nodes with non-anomalous data points (e.g., 1, 2, 4, etc.) may be processed. For example, all nodes marked with X may be processed, all nodes marked with X and child nodes directly connected to nodes marked with X may be processed, or another configuration of nodes with non-anomalous data points may be processed. In one implementation, only nodes in an isolation tree with non-anomalous data points from a first split condition are processed.

[0029] Table 1 provides a computational efficiency comparison between constructing a complete isolation tree (such as isolation tree 204) and a sparse isolation tree (such as sparse isolation tree 304). As shown in Table 1, the time spent constructing a sparse isolation tree is significantly reduced compared to the time spent constructing a complete isolation tree. Furthermore, as the dataset under consideration scales up, the gap between the time spent constructing a complete isolation tree and the time spent constructing a sparse isolation tree increases, thereby further enhancing the benefit as the dataset increases. Dataset size Processing time for a complete isolated tree (seconds) Processing time for sparse isolation trees (seconds) accelerate Time reduction 100,000 columns 4.7133 0.3442 13.96x 92.7% 250,000 columns 12.5861 0.8466 14.87x 93.3% 1,000,000 columns 55.9506 3.0443 18.62x 94.6% Table 1

[0030] Although Figures 2 and 3 illustrate the use of isolated trees / forests to interpret anomalies, those skilled in the art will understand that the concepts described herein can be applied to other ablation learning models beyond isolated trees / forests or other tree-like models. For example, in some implementations, model learning based on locally interpretable model-independent interpretations (LIME), locally linear models, and / or other suitable learning models can be used.

[0031] Figure 4 is a flowchart illustrating a non-limiting exemplary embodiment of a technique according to various forms of the present invention for determining the cause of an abnormal output of a process.

[0032] In block 402, an analysis computing system receives a set of measurements from a metrology system that represent abnormal outputs of the process. When constructing an isolation tree as described in Figure 3, for example, the set of measurements in block 402 may be represented by labeled leaf nodes. Measurements may include manufacturing properties associated with the outputs of a process step in a semiconductor process, such as those measured by the metrology system. As described herein, examples of measurements received from a metrology sensor of the metrology system may include, for example, one or more of the following: thickness, film uniformity (e.g., thickness uniformity within a wafer), film stress (e.g., wafer bending), feature dimensions (e.g., feature width), feature morphology (e.g., feature angle), optical parameters (e.g., refractive index), and defect profiles (e.g., particles). Measurements received by the computing system in block 402 include abnormal outputs. Outputs from process steps may be identified as abnormal by the analysis computing system, for example, based on a physical property or an electrical property measured using a sensor mounted on a metrology or test equipment.

[0033] In block 404, the analysis system retrieves a set of measurements from previous outputs. Previous outputs may be the outputs of the measurement system in the step leading to (but not including) the step of collecting the set of measurements from the abnormal outputs of block 402. When constructing an isolation tree as described in FIG3, for example, the measurements from previous outputs in block 404 may be represented by a parent node leading to a labeled leaf node. The measurements from previous outputs may also include manufacturing attributes associated with other outputs from process steps in the semiconductor process (e.g., manufacturing attributes associated with other non-abnormal semiconductor wafers). In some embodiments, the analysis system may retrieve a set of measurements from a data storage of previous measurements and may retrieve related previous measurements (e.g., measurements that share a characteristic with the set of measurements from abnormal outputs, such as using the same recipe, collected matching data, the same worker operating a manufacturing apparatus 108, etc.). The timely creation of isolated trees / forests with relevant data is feasible due to the increased speed provided by the technology disclosed in this paper, whereas using previous techniques for training isolated trees / forests would have been very difficult.

[0034] In block 406, the analysis and calculation system combines the previously output measurement set with the abnormal output measurement set to create a measurement dataset, wherein each measurement set in the measurement dataset is a data point, and each data point contains a plurality of features.

[0035] In sub-constant block 408, a sub-constant (such as sub-constant 500) is executed, wherein the analysis computation system constructs at least one isolation tree model. In various embodiments, the isolation tree comprises a plurality of parent nodes and leaf nodes, wherein each parent node corresponds to a splitting condition of one of the manufacturing attributes, and the leaf nodes correspond to abnormal outputs. The splitting conditions of each of the manufacturing attributes can be determined randomly (e.g., using a machine learning model).

[0036] In some of these embodiments, to construct at least one isolation tree model, the analysis system creates two or more parent nodes based on a comparison of a first measurement of a manufacturing attribute with a threshold value. Threshold values ​​may also be randomly determined or generated. Next, the analysis system may, from further consideration, omit the parent nodes of two or more parent nodes that do not contain the first measurement and create leaf nodes (also referred to as child nodes) from the remaining parent nodes. Each leaf node may be based on a comparison of a subsequent measurement of a manufacturing attribute with a subsequent threshold value (e.g., a randomly generated subsequent threshold value). Child nodes that do not contain subsequent measurements may, from further consideration, omit them. The analysis system may repeat the creation of child nodes until all measurements in the manufacturing attribute are associated with a node of the isolation tree model.

[0037] In block 410, the analysis and processing system assigns values ​​to each node of the isolation tree model. For example, the analysis and processing system may determine the Shapley Overlay Interpretation (SHAP) value of at least one node of the isolation tree model. In block 412, the analysis and processing system determines one or more features that may be associated with a manufacturing attribute having abnormal output based on the SHAP value. The determined features may be presented on a display of the analysis and processing system or otherwise provided for controlling the manufacturing system 102.

[0038] In various embodiments, the analysis processing system may construct an isolated forest (e.g., multiple isolated tree models) for determining one or more features that may be associated with a manufacturing attribute having anomalous output. The analysis processing system may track the path length of each isolated tree model (e.g., track the number of nodes between a parent node and its last child node). The analysis processing system may continue to generate isolated tree models in the isolated forest until the path lengths (e.g., average path length) converge.

[0039] Figure 5 is a flowchart illustrating a non-limiting exemplary embodiment of a sub-formula for constructing an isolation tree model according to various forms of the present invention. In block 502, the analysis system creates a parent node that divides the measurement dataset into two or more parts based on a comparison of a feature of each data point with a threshold value. In block 504, the analysis system creates a child node of the parent node for the part of the measurement set containing anomalous outputs, without processing the other parts of the measurement set that do not contain anomalous outputs, wherein the child node divides the part of the measurement set containing anomalous outputs into two or more sub-parts.

[0040] Sub-constant 500 then proceeds to a decision block 506. If the set of measurements for the abnormal output is within a leaf node of the isolation tree model (e.g., the set of measurements for the abnormal output is in a separate child node), then the result of decision block 506 is yes, and sub-constant 500 proceeds to a completion block 508 and returns control to its caller. Otherwise, the result of decision block 506 is no, and sub-constant 500 returns to block 504 to create another child node. Additional Example I 1. A computer-implemented method for determining the cause of an abnormal output of a process, the method comprising: receiving a set of measurements of the abnormal output of the process from a measurement system by an analysis and computing system; retrieving a previously output set of measurements by the analysis and computing system; combining the previously output set of measurements with the abnormal output set of measurements by the analysis and computing system to create a measurement dataset, wherein each set of measurements in the measurement dataset is a data point, and each data point contains a plurality of features; and constructing at least one isolation tree model by the analysis and computing system performing actions, the actions including: The analysis and computation system creates a parent node that divides the measurement dataset into two or more parts based on a comparison of one feature of each data point with one of the threshold values; and the analysis and computation system creates a child node of the parent node for the part of the measurement set containing the anomalous output, without processing the other parts of the measurement set that do not contain the anomalous output, wherein the child node divides the part of the measurement set containing the anomalous output into two or more sub-parts; and the analysis and computation system repeatedly creates child nodes until a leaf node of the measurement set containing the anomalous output is created; the analysis and computation system determines the Shapley Stack Interpretation (SHAP) value of each node of the at least one isolated tree model; and the analysis and computation system determines one or more features that may cause the anomalous output based on the SHAP values. 2. The computer implementation method of Embodiment 1, wherein creating the parent node includes: selecting a feature from the features of the data points; and selecting one threshold value of the feature. 3. The computer implementation method of Embodiment 2, wherein selecting the feature from the features of the data points includes randomly selecting the feature from the features of the data points; and wherein the threshold value for selecting the feature includes randomly selecting the threshold value between a minimum value and a maximum value of the feature. 4. The computer implementation method of Embodiment 3, wherein constructing at least one isolation tree model includes constructing a plurality of isolation tree models and combining them to create an isolation forest. 5. The computer implementation method of Embodiment 1, wherein the process is a semiconductor process, and wherein the abnormal output is a wafer. 6. The computer implementation method of Embodiment 1, wherein retrieving the measurement set of previous outputs includes: retrieving the measurement set of previous outputs assumed to be non-abnormal.7. A computer implementation method as described in Embodiment 1, wherein retrieving a previously output measurement set comprises: retrieving a previously output measurement set that matches at least one feature of the abnormal output measurement set. 8. A non-transitory computer-readable medium having stored thereon computer-executable instructions, which, in response to execution by one or more processors of a computing system, cause the computing system to perform an action as described in any of Embodiments 1 to 7. 9. A computing system configured to perform a method as described in any of Embodiments 1 to 7. Additional Example II 1. A method for associating an anomalous output of a semiconductor process with a manufacturing attribute of the process, the method comprising: identifying the anomalous output among a plurality of outputs in a process step of the process; receiving a manufacturing attribute associated with each of the plurality of outputs containing the anomalous output; and constructing at least one isolation tree model using a machine learning model, including a plurality of parent nodes each corresponding to a splitting condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output. 2. The method of Example 1, wherein none of the parent nodes are associated with a manufacturing attribute that does not directly cause the anomalous output. 3. The method of Embodiment 1, wherein constructing each isolation tree model includes: creating two or more parent nodes based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value; omitting parent nodes of the two or more parent nodes that do not contain the first measurement, further considering; creating two or more child nodes from the remaining parent nodes of the two or more parent nodes based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value; omitting child nodes of the two or more child nodes that do not contain the subsequent measurement, further considering; and repeating the creation of child nodes until all measurements of the manufacturing attributes are associated with a node of the isolation tree model; and wherein the method further includes determining, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attributes associated with the anomalous output. 4. The method of Embodiment 1, wherein identifying the anomalous output includes identifying it based on a physical attribute or an electrical attribute measured using a sensor installed on a metrology or testing device. 5. The method of Embodiment 1, wherein the segmentation criteria for each of the manufacturing attributes are randomly determined by the machine learning model. 6. The method of Example 1, wherein the plurality of outputs includes previous outputs in the procedure step. 7. The method of Example 6, wherein the previous outputs are associated with manufacturing attributes assumed to be non-abnormal outputs. 8. The method of Example 3, wherein determining the one or more features that may be associated with the manufacturing attributes associated with the anomalous output includes analyzing the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent nodes and each remaining child node.9. The method of Example 3, wherein the threshold value for creating the child node and each subsequent threshold value are randomly generated. 10. The method of Example 3, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolation forest; and wherein the determination of one or more features that may be associated with the manufacturing attributes associated with the anomalous output is based on the isolation forest. 11. The method of Example 10, wherein two or more parent nodes of each of the plurality of isolation tree models forming the isolation forest are randomly determined based on one of the manufacturing attributes. 12. The method of Example 1, wherein the anomalous output is a semiconductor wafer. 13. The method of any of the above embodiments, wherein the method is further based on any of the embodiments in Additional Example 1. Additional Example III 1. A non-transitory computer-readable storage medium storing instructions that, when executed by a system of one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor manufacturing process; receive manufacturing attributes associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, including a plurality of parent nodes each corresponding to a partitioning condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output. 2. The non-transitory computer-readable storage medium of Example 1, wherein none of the parent nodes are associated with a manufacturing attribute that does not directly cause the anomalous output. 3. A non-transitory computer-readable storage medium as described in Example 1, wherein, for constructing each isolation tree model, the instructions cause one or more processors to: create two or more parent nodes based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value; omit, from further consideration, the parent nodes of the two or more parent nodes that do not contain the first measurement; create two or more child nodes from the remaining parent nodes of the two or more parent nodes based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value; omit, from further consideration, the child nodes of the two or more child nodes that do not contain the subsequent measurement; and repeat the creation of child nodes until all measurements of the manufacturing attributes are associated with a node of the isolation tree model; and wherein the instructions further cause one or more processors to determine, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attributes associated with the abnormal output. 4. A non-transitory computer-readable storage medium as in Example 1, wherein, in order to identify the abnormal output, the instructions cause one or more processors to identify it based on a physical property or an electrical property measured using a sensor mounted on a metering or testing device.5. A non-transitory computer-readable storage medium as in Example 1, wherein the segmentation criteria for each of the manufacturing attributes are randomly determined by the machine learning model. 6. A non-transitory computer-readable storage medium as in Example 1, wherein the plurality of outputs includes previous outputs in the program steps. 7. A non-transitory computer-readable storage medium as in Example 6, wherein the previous outputs are associated with manufacturing attributes assumed to be non-abnormal outputs. 8. A non-transitory computer-readable storage medium as in Example 3, wherein, in order to determine one or more features that may be associated with the manufacturing attributes associated with the anomalous output, the instructions cause one or more processors to analyze the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent node and each remaining child node. 9. A non-transitory computer-readable storage medium as in Example 3, wherein the threshold value and each subsequent threshold value used to create the child node are randomly generated. 10. A non-transitory computer-readable storage medium as described in Example 3, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolated forest; and wherein the determination that one or more features may be associated with manufacturing attributes related to the anomalous output is based on the isolated forest. 11. A non-transitory computer-readable storage medium as described in Example 10, wherein two or more parent nodes of each of the plurality of isolation tree models forming the isolated forest are randomly determined based on one of the manufacturing attributes. 12. A non-transitory computer-readable storage medium as described in Example 1, wherein the anomalous output is a semiconductor wafer. 13. A non-transitory computer-readable storage medium as described in any of the above embodiments, wherein the non-transitory computer-readable storage medium further conforms to any of the embodiments in Additional Example 1. Additional Example IV 1. A system for associating an anomalous output of a semiconductor process with a manufacturing attribute of the process, the system comprising: one or more processors; and a non-transitory computer-readable storage medium storing instructions that, when executed by the one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor process; receive a manufacturing attribute associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, including a plurality of parent nodes each corresponding to a splitting condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output. 2. The system of embodiment 1, wherein none of the parent nodes are associated with a manufacturing attribute that does not directly cause the anomalous output.3. The system of Embodiment 1, wherein, to construct the isolation tree models, the instructions cause one or more processors to: create two or more parent nodes based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value; omit the parent nodes of the two or more parent nodes that do not contain the first measurement, taking a further consideration; create two or more child nodes from the remaining parent nodes of the two or more parent nodes based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value; omit the child nodes of the two or more child nodes that do not contain the subsequent measurement, taking a further consideration; and repeat the creation of child nodes until all measurements of the manufacturing attributes are associated with a node of the isolation tree model; and wherein the instructions further cause one or more processors to determine, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attributes associated with the anomalous output. 4. The system of Embodiment 1, wherein, to identify the anomalous output, the instructions cause one or more processors to identify it based on a physical attribute or an electrical attribute measured using a sensor mounted on a metrology or testing device. 5. The system of Embodiment 1, wherein the segmentation criteria for each of the manufacturing attributes are randomly determined by the machine learning model. 6. The system of Embodiment 1, wherein the plurality of outputs includes previous outputs in the program steps. 7. The system of Embodiment 6, wherein the previous outputs are associated with manufacturing attributes assumed to be non-abnormal outputs. 8. The system of Embodiment 3, wherein, to determine one or more features that may be associated with the manufacturing attribute associated with the anomalous output, the instructions cause one or more processors to analyze the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent nodes and each remaining child node. 9. The system of Embodiment 3, wherein the threshold value and subsequent threshold values ​​used to create the child nodes are randomly generated. 10. The system of Embodiment 3, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolation forest; and wherein the determination of one or more features that may be associated with the manufacturing attribute associated with the anomalous output is based on the isolation forest. 11. The system of Example 10, wherein two or more parent nodes of each of the plurality of isolation tree models forming the isolation forest are randomly determined based on one of the manufacturing attributes. 12. The system of Example 1, wherein the abnormal output is a semiconductor wafer. 13. The system of any of the above embodiments, wherein the system further conforms to any of the embodiments in Additional Example 1.

[0041] Although illustrative embodiments have been drawn and described, it should be understood that various changes may be made herein without departing from the spirit and scope of the invention.

[0042] Unless otherwise expressly required herein, throughout the description and embodiments, the words "comprising," "including," and similar terms shall be construed as encompassing rather than exclusive or exhaustive; that is, they shall be construed as "including, but not limited to." Furthermore, the words "in this document," "above," "below," and similar terms, when used in this application, shall refer to the entire application and not any particular part thereof. Where permitted by the text, singular or plural terms used in the detailed description may also include either the plural or the singular, respectively. The word "or" refers to one of two or more items in a list, and is intended to encompass all of the following interpretations: any item in the list, all items in the list, or any combination of items in the list. All numerical values ​​provided herein are intended to include similar values ​​within a measurement error range.

[0043] Furthermore, unless otherwise expressly stated or as otherwise understood in the text, the conditional language used herein, especially words such as “may,” “possibly,” “can,” “able,” “for example,” “like,” and the like, are generally intended to express that certain embodiments include (but not other embodiments) certain features, elements, and / or states.

[0044] The teachings provided herein can be applied to other systems, not necessarily the systems described above. The elements and actions of the various embodiments described above can be combined to provide further embodiments. The actions of the methods discussed herein can be performed in any suitable manner. Furthermore, the actions of the methods discussed herein can be performed in series or in parallel as appropriate.

[0045] Although certain embodiments have been described, these embodiments are presented by way of example only and are not intended to limit the scope of the invention. In fact, the novel methods and systems described herein can be embodied in various other forms. Furthermore, various omissions, substitutions, and changes in the form of the methods and systems described herein can be made without departing from the spirit of the invention. For example, although the disclosed embodiments are presented in a given configuration, alternative embodiments may perform similar functionality with different components and / or circuit topologies, and some elements may be deleted, moved, added, subdivided, combined, and / or modified. Each of these elements may be implemented in various suitable different ways. Any suitable combination of elements and actions of the various embodiments described above may be combined to provide further embodiments. The appended claims and their equivalents are intended to cover such forms or modifications that fall within the scope and spirit of the invention. Therefore, the scope of the invention is defined by reference to the appended claims. [Simplified Explanation of the Diagram]

[0009] Embodiments of the present invention will be described by way of example with reference to the accompanying drawings.

[0010] Figure 1 is a schematic diagram of a computing system for analyzing a semiconductor manufacturing process according to various forms of the present invention.

[0011] Figure 2 shows an example of one of the machine learning techniques that can be used to detect abnormal data.

[0012] Figure 3 shows an example of another machine learning technique that can be used to detect anomalous data.

[0013] Figure 4 is an illustrative procedure for determining the cause of abnormal output in a process according to various states according to the present invention.

[0014] Figure 5 is an example procedure of a sub-convention for constructing an isolation tree model according to various states of the present invention.

Claims

1. A method for associating an abnormal output of a semiconductor process with a manufacturing attribute of the process, the method comprising: In one process step of the manufacturing process, the abnormal output among a plurality of outputs is identified; a manufacturing attribute associated with each of the plurality of outputs containing the abnormal output is received; and a machine learning model is used to construct at least one isolation tree model, which includes a plurality of parent nodes corresponding to a splitting condition of one of the manufacturing attributes and a leaf node corresponding to the abnormal output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the abnormal output.

2. As in request item 1, where none of these parent nodes are associated with manufacturing properties that do not directly cause the abnormal output.

3. As in request item 1, the method for constructing each isolation tree model includes: Based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value, two or more parent nodes are created; further considering, parent nodes of the two or more parent nodes that do not contain the first measurement are omitted; based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value, two or more child nodes are created from the remaining parent nodes of the two or more parent nodes; further considering, child nodes of the two or more child nodes that do not contain the subsequent measurement are omitted; and the creation of child nodes is repeated until all measurements of the manufacturing attributes are associated with a node of the isolation tree model; and the method further includes determining, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attributes associated with the anomalous output.

4. The method of claim 1, wherein identifying the anomalous output includes identifying it based on a physical property or an electrical property measured using a sensor mounted on a metering or testing device.

5. The method of request item 1, wherein the segmentation condition for each of the manufacturing attributes is randomly determined by the machine learning model.

6. The method of request item 1, wherein the plurality of outputs includes previous outputs in the procedure step.

7. The method of request item 6, wherein such previous outputs are associated with manufacturing attributes that are assumed to be non-abnormal outputs.

8. The method of request item 3, wherein determining the one or more features that may be associated with the manufacturing attributes associated with the anomalous output includes analyzing the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent node and each remaining child node.

9. As in request item 3, wherein the threshold value used to create the child node and each subsequent threshold value are randomly generated.

10. The method of claim 3, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolation forest; and wherein the one or more features that may be associated with the manufacturing attributes associated with the anomalous output are based on the isolation forest.

11. The method of claim 10, wherein two or more parent nodes of each of the plurality of isolated tree models forming the isolated forest are randomly determined based on one of the manufacturing attributes.

12. The method of request item 1, wherein the abnormal output is a semiconductor wafer.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a system of one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor manufacturing process; receive a manufacturing attribute associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, including a plurality of parent nodes each corresponding to a partitioning condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output.

14. Non-transitory computer-readable storage media as in request item 13, wherein none of such parent nodes are associated with manufacturing attributes that do not directly cause the abnormal output.

15. The non-transitory computer-readable storage medium of claim 13, wherein, for constructing the isolation tree models, the instructions cause one or more processors to: create two or more parent nodes based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value; omit, from further consideration, the parent nodes of the two or more parent nodes that do not contain the first measurement; create two or more child nodes from the remaining parent nodes of one of the two or more parent nodes based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value; omit, from further consideration, the child nodes of the two or more child nodes that do not contain the subsequent measurement; and repeat the creation of child nodes until all measurements in the manufacturing attributes are associated with a node of the isolation tree model; and wherein the instructions further cause one or more processors to determine, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attributes associated with the abnormal output.

16. The non-transitory computer-readable storage medium of claim 13, wherein, in order to identify the anomalous output, the instructions cause one or more processors to identify it based on a physical or electrical property measured using a sensor mounted on a metering or testing device.

17. The non-transitory computer-readable storage medium of claim 13, wherein the segmentation criteria for each of the manufacturing attributes are randomly determined by the machine learning model.

18. A non-transitory computer-readable storage medium as requested in claim 13, wherein the plurality of outputs includes previous outputs in the program steps.

19. The non-transitory computer-readable storage medium as in claim 18, wherein such prior output is associated with manufacturing attributes assumed to be non-abnormal output.

20. The non-transitory computer-readable storage medium of claim 15, wherein, in order to determine one or more features that may be associated with the manufacturing properties associated with the anomalous output, the instructions cause one or more processors to analyze the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent node and each remaining child node.

21. The non-transitory computer-readable storage medium as requested in item 15, wherein the threshold value and subsequent threshold values ​​used to create child nodes are randomly generated.

22. The non-transitory computer-readable storage medium of claim 15, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolation forest; and wherein the one or more features that may be associated with manufacturing attributes related to the anomalous output are based on the isolation forest.

23. The non-transitory computer-readable storage medium of claim 22, wherein two or more parent nodes of each of the plurality of isolated tree models forming the isolated forest are randomly determined based on one of the manufacturing attributes.

24. The non-transitory computer-readable storage medium of claim 13, wherein the abnormal output is a semiconductor wafer.

25. A system for associating an abnormal output of one of a semiconductor manufacturing processes with a manufacturing attribute of that process, the system comprising: One or more processors; and a non-transitory computer-readable storage medium, storing instructions that, when executed by one or more processors, cause the one or more processors to: identify an anomalous output among a plurality of outputs in a program step of a semiconductor process; receive manufacturing attributes associated with each of the plurality of outputs containing the anomalous output; and construct at least one isolation tree model using a machine learning model, including a plurality of parent nodes each corresponding to a partitioning condition of one of the manufacturing attributes and a leaf node corresponding to the anomalous output, wherein each of the parent nodes of the at least one isolation tree model is associated with a manufacturing attribute that directly causes the anomalous output.

26. The system as requested in item 25, wherein none of these parent nodes are associated with manufacturing attributes that do not directly cause the abnormal output.

27. The system of claim 25, wherein, for constructing each isolation tree model, the instructions cause one or more processors to: create two or more parent nodes based on a comparison of a first measurement of one of the manufacturing attributes with a threshold value; omit, from further consideration, the parent nodes of the two or more parent nodes that do not contain the first measurement; create two or more child nodes from the remaining parent nodes of the two or more parent nodes based on a comparison of a subsequent measurement of one of the manufacturing attributes with a subsequent threshold value; omit, from further consideration, the child nodes of the two or more child nodes that do not contain the subsequent measurement; and repeat the creation of child nodes until all measurements of the manufacturing attributes are associated with a node of the isolation tree model; and wherein the instructions further cause one or more processors to determine, based on the at least one isolation tree model, one or more features that may be associated with the manufacturing attribute associated with the abnormal output.

28. The system of request item 25, wherein, in order to identify the abnormal output, the instructions cause one or more processors to identify it based on a physical property or an electrical property measured using a sensor mounted on a metering or testing device.

29. The system of request item 25, wherein the segmentation condition of each of the manufacturing attributes is randomly determined by the machine learning model.

30. The system as requested in item 25, wherein the plurality of outputs includes previous outputs in the procedure steps.

31. The system of request item 30, wherein such previous outputs are associated with manufacturing attributes that are assumed to be non-abnormal outputs.

32. The system of request 27, wherein, in order to determine one or more features that may be associated with the manufacturing attributes associated with the abnormal output, the instructions cause one or more processors to analyze the Shapley Overlay Interpretation (SHAP) values ​​of the remaining parent node and each remaining child node.

33. The system as requested in item 27, wherein the threshold value used to create child nodes and each subsequent threshold value are randomly generated.

34. The system of claim 27, wherein the at least one isolation tree model is a plurality of isolation tree models forming an isolation forest; and wherein the one or more features that may be associated with the manufacturing attributes associated with the anomalous output are based on the isolation forest.

35. The system of claim 34, wherein two or more parent nodes of each of the plurality of isolated tree models forming the isolated forest are randomly determined based on one of the manufacturing attributes.

36. The system of request item 25, wherein the abnormal output is a semiconductor wafer.

Images