Method, device, and storage medium for determining cause of poor performance

Abstract

The application discloses a method, device and storage medium for determining a cause of defects, the method comprising: obtaining equipment characteristic data of a production process of a target product; training at least two prediction models using the equipment characteristic data, and extracting first characteristic importance of N-dimensional equipment characteristics from each trained prediction model to obtain at least two groups of first characteristic importance, each group of first characteristic importance representing a contribution degree of the N-dimensional equipment characteristics to generation of defective products in the production process of the target product; determining second characteristic importance of the N-dimensional equipment characteristics according to the at least two groups of first characteristic importance; and determining cause information of generation of defective products in the production process of the target product from the N-dimensional equipment characteristics according to the second characteristic importance. The application solves the technical problem that the accuracy of a cause of generation of defective products obtained by analyzing production process data of a product by using a computer is not high.

Description

Technical Field

[0001] This invention belongs to the field of manufacturing, and particularly relates to a method, apparatus and storage medium for determining the cause of defects. Background Technology

[0002] In the manufacturing industry, product quality control is a crucial link in ensuring product reliability and consistency. Traditional quality control methods rely on manual inspection and experience-based judgment, which is not only inefficient but also easily influenced by subjective factors. With the development of technology, using computers to process and analyze data obtained in manufacturing, replacing manual reasoning, judgment, and decision-making to identify and predict quality problems, has become an important means to improve production efficiency and product quality.

[0003] Because the amount of production process data recorded throughout the entire product manufacturing process is enormous, and the sample size of the same batch of products is relatively small, the accuracy of the defects obtained by analyzing the production process data using computers is not high. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for determining the cause of defects, thereby solving the technical problem that the accuracy of the causes of defects obtained by analyzing product production process data using computers is not high.

[0005] According to a first aspect of the present invention, a method for determining the cause of defects is provided, comprising: acquiring equipment feature data of the production process of a target product, the equipment feature data including feature values ​​of N-dimensional equipment features of the process equipment used to produce the target product, where N is an integer greater than 1; training at least two prediction models using the equipment feature data, and extracting a first feature importance of the N-dimensional equipment features from each of the trained prediction models to obtain at least two sets of first feature importance, each set of first feature importance characterizing the degree of contribution of the N-dimensional equipment features to the generation of defects in the production process of the target product; determining a second feature importance of the N-dimensional equipment features based on the at least two sets of first feature importance; and determining the cause information of defects generated in the production process of the target product from the N-dimensional equipment features based on the second feature importance.

[0006] In conjunction with the first aspect, in some embodiments, obtaining the equipment feature data of the production process of the target product includes: obtaining production process data of M batches of the target product, where M is a positive integer; performing multi-level filtering on the production process data of the M batches to obtain the equipment feature data, wherein the equipment feature data includes multiple training samples, and each training sample has feature values ​​of various dimensions of equipment features related to the production of defective products when producing one piece of the target product.

[0007] In conjunction with the first aspect, in some embodiments, obtaining the production process data of M batches of the target product includes: recording and storing full production process data in a database throughout the entire production process of the target product, wherein the full production process data includes the parameter values ​​of the equipment parameters of the process equipment used to produce each target product; after the production of M batches of the target product is completed, obtaining the defect rate of the M batches of the target product; if the defect rate of the M batches of the target product is greater than a preset threshold, importing the production process data of the M batches of the target product from the database.

[0008] In conjunction with the first aspect, in some embodiments, the production process data of the M batches are subjected to multi-level screening to obtain the equipment feature data, including: performing a first-level data screening on the production process data through data preprocessing to obtain a first dataset; performing a second-level data screening on the first dataset through feature engineering to obtain a second dataset; inputting the second dataset into a pre-trained anomaly detection model, and performing anomaly sample screening on the second dataset through the anomaly detection model to obtain the equipment feature data.

[0009] In conjunction with the first aspect, in some embodiments, training at least two prediction models using the device feature data includes: performing device feature selection on the device feature data to obtain a training dataset, wherein the training dataset includes multiple training samples, each training sample having multi-dimensional device features, the correlation between the multi-dimensional device features and the production of defective products being higher than the correlation between the various-dimensional device features that were removed during the device feature selection and the production of defective products, and the N-dimensional device features belonging to the multi-dimensional device features; inputting the training dataset into the at least two prediction models to train the at least two prediction models, wherein during the training process of each prediction model, the prediction model is used to predict whether a corresponding target product is a good product based on the training samples.

[0010] In conjunction with the first aspect, in some embodiments, the training samples input to the at least two prediction models are the same.

[0011] In conjunction with the first aspect, in some embodiments, before inputting the training dataset into the at least two prediction models and training the at least two prediction models, the method further includes: determining the target hyperparameters of each of the at least two prediction models; and initializing the prediction model according to the target hyperparameters of the prediction model for each of the prediction models.

[0012] In conjunction with the first aspect, in some embodiments, the at least two prediction models include a first decision tree model and a second decision tree model; the step of inputting the training dataset into the at least two prediction models to train the at least two prediction models includes: inputting the training dataset into the first decision tree model for training, and inputting the training dataset into the second decision tree model for training, extracting a set of first feature importance from the trained first decision tree model, and extracting another set of first feature importance from the trained second decision tree model; wherein, the model structure of the first decision tree model is different from the model structure of the second decision tree model, and each set of first feature importance includes a first importance score of the N-dimensional device feature, the first importance score representing the degree of contribution of the corresponding dimension device feature to the prediction result, and the prediction result being the result of predicting whether the target product is a good product.

[0013] In conjunction with the first aspect, in some embodiments, determining the second feature importance of the N-dimensional device features based on the at least two sets of first feature importance includes: testing each of the trained prediction models on a test dataset to obtain the prediction accuracy of each of the trained prediction models; assigning weight coefficients to the at least two prediction models based on the prediction accuracy of each of the trained prediction models; and performing a weighted average of the at least two sets of first feature importance based on the weight coefficients of each prediction model to obtain the second feature importance.

[0014] In conjunction with the first aspect, in some embodiments, the second feature importance includes the second importance score of each dimension of the N-dimensional equipment features, where the second importance score of each dimension of the equipment feature represents the degree of contribution of that dimension of equipment feature to the production of defective products in the target product after weighted averaging. The step of determining the cause information of defective products generated in the production process of the target product from the N-dimensional equipment features based on the second feature importance includes: selecting K-dimensional equipment features from the N-dimensional equipment features that exceed a preset score threshold based on the second importance score of each dimension of the second feature importance, where K is a positive integer less than or equal to N; and locating the cause information of defective products generated in the production process of the target product to the K-dimensional equipment features.

[0015] In conjunction with the first aspect, in some embodiments, determining the cause information of defective products generated during the production process of the target product from the N-dimensional equipment features based on the importance of the second feature further includes: obtaining historical data of the process equipment corresponding to the K-dimensional equipment features; and determining the cause information of defective products generated during the production process of the target product from the K-dimensional equipment features based on the historical data.

[0016] In a second aspect of the invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method for determining the cause of a malfunction as described in any embodiment of the first aspect.

[0017] In a third aspect of the invention, a computer-readable storage medium is provided that stores computer instructions for causing the computer to perform the method for determining the cause of a malfunction as described in any embodiment of the first aspect.

[0018] According to one or more embodiments provided in this invention, at least the following technical effects or advantages are achieved:

[0019] Because the equipment parameters of the process equipment used in the production of the target product are dynamically changing, and the equipment status, environmental conditions, and operating parameters will differ in different batches of the target product, the data types and distributions related to defective products will also change dynamically. Different prediction models have different sensitivities to different types and distributions of data. By training at least two prediction models using equipment feature data from the production process of the target product, and extracting the first feature importance of N-dimensional equipment features from each trained prediction model, and then combining the first feature importance of the N-dimensional equipment features from each of the at least two prediction models to determine the second feature importance of the N-dimensional equipment features, the cause information of defective products in the production process of the target product can be determined from the aforementioned N-dimensional equipment features. This approach can address the differences in the data types and distributions related to defective products in different batches of the target product, thereby reducing the potential bias in the training results of a single prediction model, improving the accuracy of feature importance assessment, and thus improving the accuracy of the analyzed cause information of defective products.

[0020] The above description is merely an overview of the technical solutions provided by some embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the embodiments of the present invention more obvious and easy to understand, specific implementation methods of the embodiments of the present invention are described below. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1A flowchart of a method for determining the cause of a defect in some embodiments of the present invention is shown;

[0023] Figure 2 It shows Figure 1 An exemplary process for determining the cause of a defect;

[0024] Figure 3 A schematic diagram of the structure of an electronic device in some embodiments of the present invention is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0027] The method for determining the cause of defects provided in this invention can be applied to the manufacturing of display panels or other products. Figure 1 A flowchart illustrating a method for determining the cause of a defect in some embodiments of the present invention is shown. For example... Figure 1 As shown, the method for determining the cause of failure includes the following steps S101 to S104.

[0028] S101: Obtain equipment characteristic data of the production process of the target product. The equipment characteristic data includes the feature values ​​of the N-dimensional equipment characteristics of the process equipment used to produce the target product.

[0029] The target product can be a display panel, such as a liquid crystal display (LCD) panel or an organic light-emitting diode (OLED) display panel.

[0030] In some embodiments, the equipment feature data can directly use the production process data of M batches of the target product, where M is a positive integer. The production process data of the M batches includes the parameter values ​​of the equipment parameters of the process equipment used when producing each of the M batches of the target product. That is, the production process data of the M batches is directly used as the training dataset. Each training sample in the training dataset has the parameter values ​​of various equipment parameters of the process equipment used when producing one target product. At this time, the parameter values ​​of various equipment parameters of the process equipment of each training sample are equivalent to the feature values ​​of each dimension of the equipment features of that training sample.

[0031] In other embodiments, obtaining equipment feature data of the production process of the target product may include: obtaining production process data of M batches of the target product, where M is a positive integer; performing multi-level filtering on the production process data of the M batches to obtain equipment feature data; using the equipment feature data as a training dataset, where each training sample in the training dataset has a feature value of each dimension of equipment feature related to the production of defective products when producing one target product.

[0032] Because the amount of production process data recorded throughout the entire production process of the target product is substantial, directly inputting this data into the predictive model for training would hinder the model's efficiency in identifying the various equipment characteristics that lead to defective products. However, by pre-screening the production process data through multiple levels and using the selected equipment characteristic data to train at least two predictive models, even with a massive amount of production process data, the predictive models can quickly and accurately identify the various equipment characteristics that lead to defective products. This allows for the accurate and rapid identification of the root causes of defective products during the production process.

[0033] It should be noted that the production process data for the M batches includes the parameter values ​​of various equipment used in the production of each target product within the M batches. Taking a display panel as an example, the equipment parameters include equipment identification and at least one of the following: operating status parameters, operational parameters, and environmental parameters. It should be understood that the equipment identification is a parameter used to uniquely identify the equipment, such as equipment number or model number. The production process of a display panel involves multiple pieces of equipment, including lithography machines, etching machines, cleaning machines, and dryers. The data records the parameter values ​​of various equipment throughout the entire display panel production process, such as: exposure time, exposure intensity, and alignment accuracy for the lithography machine; etching time, etching rate, and gas flow rate for the etching machine; cleaning fluid temperature, cleaning pressure, and cleaning time for the cleaning machine; and drying temperature and drying time for the dryer. Operating status parameters reflect the operating status of the equipment, such as temperature, pressure, flow rate, current, and voltage. Operational parameters are adjustable parameters during equipment operation, such as speed, power, time, and position. Environmental parameters are parameters related to the operating environment of the equipment, such as ambient temperature, ambient humidity, and environmental cleanliness.

[0034] In some embodiments, obtaining production process data for M batches of the target product may include: recording and storing full production process data in a database throughout the entire production process of the target product; the full production process data includes parameter values ​​for each type of equipment used in the production of each of the M batches of the target product; after the production of the M batches of the target product is completed, obtaining the defect rate of the M batches of the target product; if the defect rate of the M batches of the target product is greater than a preset threshold, importing the production process data of the M batches of the target product from the database to determine the cause information of defective products for the M batches of the target product. If the defect rate of the M batches of the target product is less than or equal to the preset threshold, it is not necessary to determine the cause information of defective products for the M batches of the target product. Since the defect rate of the target product is less than or equal to the preset threshold, it is basically determined that the defect is not caused by equipment, and it is not necessary to import the production process data of the most recent M batches from the database to find the equipment characteristics that caused the defective products.

[0035] By automatically importing the production process data of the M batches from the database when the defect rate exceeds the standard, fully automated monitoring and analysis are achieved, improving the efficiency and accuracy of quality control in the manufacturing process.

[0036] In some embodiments, M=1, that is, in step S101, after each batch of target products is produced, the equipment characteristic data of the production process of the current batch of target products (that is, the most recently produced batch of target products) is obtained, which is used to locate the cause of defects in the current batch of target products. This enables real-time monitoring and rapid response in the manufacturing scenario of the target products. When the defect rate exceeds the standard, the production process data of the current batch is analyzed immediately. Targeted analysis is performed on each batch with an excessive defect rate, which can promptly identify the root cause of the excessive defect rate in the current batch, so as to take timely measures to process the equipment parameters and avoid affecting the production quality of subsequent batches.

[0037] In some embodiments, by performing multi-level screening on the production process data of M batches of the target product, various equipment features unrelated to the generation of defective products can be eliminated. This is beneficial for the prediction model to quickly and accurately identify the equipment features that cause defective products when the amount of production process data is large.

[0038] In some embodiments, production process data of M batches of the target product can be subjected to a three-level data filtering process to obtain equipment feature data. It is understood that the three-level data filtering process may include: performing a first-level data filtering on the production process data through data preprocessing to obtain a first dataset; performing a second-level data filtering on the first dataset through feature engineering to obtain a second dataset; and inputting the second dataset into a pre-trained anomaly detection model to perform anomaly sample filtering on the second dataset using the anomaly detection model to obtain equipment feature data.

[0039] Understandably, preprocessing the production process data of M batches of the target product can clean the collected raw data, transforming it into a first dataset that can be used for subsequent data analysis, thus providing accurate input for the analysis.

[0040] In some embodiments, data preprocessing of the production process data of M batches of the target product may include: processing the parameter values ​​of the equipment parameters in the production process data of the M batches of the target product that are missing, duplicate, or outlier. When processing missing values, mean imputation, median imputation, or interpolation methods can be used to replace missing values ​​with other values ​​to avoid the incompleteness of the data affecting the analysis results. Duplicate data is identified and merged or deleted. When processing outlier data, the standard deviation of multiple parameter values ​​of the same dimension of the equipment parameter (e.g., cleaning time collected at different times) can be calculated to identify parameter values ​​whose difference from the standard deviation is greater than a preset threshold, and these values ​​can be corrected or removed.

[0041] Understandably, performing a second level of data filtering on the first dataset through feature engineering can include: converting the format and type of each device parameter in the first dataset, and then normalizing the converted data to transform it into the second dataset. Specifically, when converting the format and type of the device parameters in the first dataset, non-numerical parameters such as categorical parameters, text parameters, and time parameters can be encoded to convert their values ​​into numerical features. Depending on the specific needs, one-hot encoding or label encoding can be used to convert categorical parameters into numerical features.

[0042] In some embodiments, when screening for anomalous samples using a pre-trained anomaly detection model, the anomaly detection model can be a pre-trained Isolation Forest model.

[0043] The second dataset is input into the Isolation Forest model. This dataset includes multiple optional samples; one optional sample corresponds to one target product in M ​​batches. Each optional sample includes feature values ​​of multi-dimensional equipment characteristics for each process device used to produce that target product. For each optional sample, the Isolation Forest model calculates an anomaly score based on the feature values ​​of its multi-dimensional equipment characteristics. The anomaly score represents the probability or degree to which the optional sample is identified as an anomaly; a higher anomaly score indicates a greater likelihood of the optional sample being an anomaly. Based on a pre-set score threshold, optional samples with anomaly scores higher than the threshold are identified as anomalies. For example, if the score threshold is set to 0.7, then optional samples with anomaly scores greater than 0.7 are marked as anomalies. Selecting optional samples from the second dataset that exceed the pre-set score threshold serves as equipment feature data for training each prediction model. This allows for a rough identification of the range of equipment features causing defects, reducing the amount of equipment feature data input to the prediction model and effectively identifying anomalous equipment features even with high equipment feature dimensions and a small number of optional samples.

[0044] By combining the above-mentioned feature process with the anomaly detection model, we can roughly locate the range of equipment features that lead to defective products, extract the equipment features related to the generation of defective products, that is, more evaluable equipment features, so that each prediction model can better evaluate the contribution of each dimension of equipment features related to the generation of defective products to the generation of defective products, and eliminate the interference of irrelevant equipment features.

[0045] S102: Train at least two prediction models using equipment feature data, and extract the first feature importance of N-dimensional equipment features from each prediction model after training to obtain at least two sets of first feature importance, each set of first feature importance representing the degree of contribution of N-dimensional equipment features to the production of defective products in the target product.

[0046] Understandably, the contribution of N-dimensional equipment features to the production of defective products in the target product process stems from the contribution of these N-dimensional equipment features to the prediction results of the prediction model. Each prediction model's result is its prediction of whether each target product in M ​​batches is a good product. The importance of each group of first features includes the first importance score of the N-dimensional equipment features, with the first importance score of each feature characterizing its contribution to the prediction results of the prediction model.

[0047] In some embodiments, when training at least two prediction models using device feature data, the obtained device feature data can be used as a training dataset and input into at least two prediction models to train the at least two prediction models.

[0048] In other embodiments, training at least two prediction models using equipment feature data may include: performing equipment feature selection on the equipment feature data to obtain a training dataset, wherein one training sample in the training dataset corresponds to one target product, and each training sample in the training dataset has feature values ​​of multi-dimensional equipment features of the process equipment that produces one target product, wherein the correlation between the multi-dimensional equipment features and the defective products generated during the production process of the target product is higher than the correlation between the various equipment features that were removed during feature selection and the defective products generated during the production process of the target product; and inputting the training dataset into at least two prediction models to train at least two prediction models respectively.

[0049] In some embodiments, feature selection can be performed on equipment feature data using any of the feature selection methods such as filtering, packaging, or embedding. This allows for the selection of multi-dimensional equipment features with a high degree of correlation to the production of defective products from various dimensions of equipment features related to defective products. As a result, the multi-dimensional equipment features with a higher degree of correlation to the production of defective products in each training sample are retained, while other equipment features with a lower degree of correlation to defective products are removed. This further optimizes the training samples used to train the prediction model, reduces the feature dimensions of each training sample used to train the prediction model, and improves analysis efficiency.

[0050] It should be noted that in the training dataset, the N-dimensional equipment features of different training samples have the same feature categories. For example, the equipment parameters involved in the production process of each target product in the same batch include the equipment features of multiple process equipment such as lithography machines, etching machines, cleaning machines, and dryers. These include: exposure time, exposure intensity, and alignment accuracy of the lithography machine; etching time, etching rate, and gas flow rate of the etching machine; cleaning fluid temperature, cleaning pressure, and cleaning time of the cleaning machine; and drying temperature and drying time of the dryer. Taking N=5 as an example, after feature selection of the equipment feature data, the following 5-dimensional equipment features can be obtained: exposure time and exposure intensity of the lithography machine; etching time of the etching machine; cleaning fluid temperature of the cleaning machine; and drying temperature of the dryer. Therefore, each training sample in the training dataset possesses the above 5-dimensional equipment features.

[0051] In some embodiments, the training samples input into at least two different prediction models are identical. That is, the same training sample is input into different prediction models to iteratively train each prediction model until the prediction accuracy of each prediction model reaches a certain preset accuracy threshold or the number of iterations reaches a preset number, thereby ensuring fair comparison between different prediction models and the reliability of the results.

[0052] In some embodiments, before inputting the training dataset into at least two prediction models and training the at least two prediction models, the method may further include: determining the target hyperparameters for each of the at least two prediction models; and initializing each prediction model according to its target hyperparameters. The training dataset is then input into each prediction model after initialization for training.

[0053] It is understandable that the target hyperparameters represent the optimal combination of hyperparameters. The hyperparameters of each prediction model can be optimized using any of the following methods: grid search, random search, Bayesian optimization, etc., to obtain the optimal hyperparameter combination for each prediction module. It should be noted that the specific optimization method used for hyperparameter optimization can be determined based on the specific training dataset and the production / manufacturing scenario.

[0054] In some embodiments, optimizing the hyperparameters of each prediction model using grid search may include: for each prediction model, selecting the types of hyperparameters that need to be adjusted and setting the value range for each hyperparameter type; generating all possible combinations of hyperparameters based on the value ranges of each hyperparameter type; evaluating the performance of the prediction model for each possible combination of hyperparameters using cross-validation (e.g., K-fold cross-validation) to obtain performance metrics (e.g., accuracy, mean squared error) for each hyperparameter combination; comparing the performance metrics of all possible hyperparameter combinations and selecting the hyperparameter combination with the best performance metrics as the target hyperparameters for that prediction model.

[0055] By selecting the optimal combination of hyperparameters to initialize the prediction model to be trained, the prediction model can achieve the highest prediction accuracy after training, thereby improving the accuracy of the first feature importance of the N-dimensional device features extracted from the trained prediction model.

[0056] It is understandable that at least two of the above prediction models can use tree models or neural network models, which can extract feature importance after training.

[0057] For example, one of the prediction models used is the eXtreme GradientBoosting (XGBoost) model, and the types of hyperparameters that need to be adjusted include the learning rate, the number of trees (n_estimators), the maximum depth of each tree (max_depth), and the proportion of data samples used to train each tree (subsample).

[0058] For example, one of the prediction models used is the Light-Gradient Boosting Machine (LightGBM) model. The types of hyperparameters that need to be adjusted include the learning rate; the number of trees (n_estimators), the maximum depth of each tree (max_depth), the proportion of data samples used to train each tree (subsample), and the proportion of feature samples used by each tree (colsample_bytree).

[0059] In some embodiments, the at least two prediction models include a first decision tree model and a second decision tree model. Training the at least two prediction models by inputting the training dataset into them includes: inputting the training dataset into the first decision tree model for training, and inputting the training dataset into the second decision tree model for training; extracting a set of first feature importance from the trained first decision tree model, and extracting another set of first feature importance from the trained second decision tree model, to obtain two sets of second feature importance.

[0060] Understandably, the model structure of the first decision tree model is different from that of the second decision tree model, and it is able to cope with device feature data with different data distributions and data types.

[0061] The first feature importance extracted from the first decision tree model includes the first importance score of the N-dimensional device features. Each first importance score represents the degree of contribution of the corresponding device feature to the prediction result of the first decision tree model. The prediction result is the result of the first decision tree model predicting whether the target product is a good product.

[0062] The first feature importance extracted from the second decision tree model includes the first importance score of the N-dimensional device features. Each first importance score represents the degree of contribution of the corresponding device feature to the prediction result of the second decision tree model. The prediction result is the result of the second decision tree model predicting whether the target product is a good product.

[0063] For example, in the first decision tree model and the second decision tree model, one is an XGBoost model and the other is a LightGBM model.

[0064] For example, for the XGBoost model, a hyperparameter combination that optimizes performance metrics can be selected through hyperparameter optimization as follows: learning_rate: 0.1 (learning rate, controlling the contribution of each tree); n_estimators: 100 (number of trees); max_depth: 6 (maximum depth of each tree); subsample: 0.8 (proportion of data samples used to train each tree).

[0065] For example, for the LightGBM model, a hyperparameter combination that optimizes the performance metrics can be: learning_rate: 0.01; n_estimators: 100; max_depth: -1 (unrestricted tree depth, allowing it to grow to its maximum); subsample: 0.9; colsample_bytree: 0.8.

[0066] It should be noted that the values ​​of the above hyperparameter combinations are for illustrative purposes only, and the specific values ​​should be based on the optimization results.

[0067] In some embodiments, the process of training the initialized XGBoost model and the initialized LightGBM model using the acquired device feature data may include the following steps one and two:

[0068] Step 1: Input the training dataset into the XGBoost model and iteratively train the XGBoost model until one of the following conditions is met: 1. The prediction accuracy of the XGBoost model reaches the preset accuracy threshold. 2. The number of iterations reaches the preset number, specifically: the number of trees in the XGBoost model reaches the number of trees specified by the hyperparameter n estimators of the XGBoost model;

[0069] In step one, the objective function of the XGBoost model is as follows:

[0070]

[0071] γ i y represents the weight of the i-th training sample, used to handle imbalanced data or weighted samples; i f is the true value of the i-th training sample; k The sum of the output values ​​of all leaf nodes in the k-th tree; n is the number of samples used in the current iteration; the XGBoost model is an ensemble of multiple trees, and K is the number of trees in the XGBoost model; I k Let represent the set of all leaf nodes of the k-th tree; λ is the coefficient of the L2 regularization term for the leaf nodes, used to control the complexity of the output values ​​of the leaf nodes; γ is the coefficient of the L1 regularization term for the leaf nodes, used to control the number of leaf nodes; x i Let θ be the i-th training sample; k θk represents the parameters of the k-th tree, including the output values ​​of the leaf nodes; gk(xi,θk) is the gradient statistic of the k-th tree, which is usually the first derivative of the loss function with respect to the output of the k-th tree.

[0072] The objective function of the XGBoost model controls model complexity by adding a regularization term, which helps prevent overfitting and improves the model's generalization ability.

[0073] The objective function of the XGBoost model consists of a loss function and a regularization term. The loss function... Calculate the loss of the training samples used in the current iteration, where γ i and y iThese are the weights and true labels of the training samples, respectively. The goal of this part is to minimize the difference between the XGBoost model's predictions and the true values. The regularization part includes two regularization terms: an L2 regularization term, used to control the complexity of the leaf node output values ​​to prevent overfitting; and an L1 regularization term, used to control the number of leaf nodes to further prevent overfitting.

[0074] The objective function of XGBoost, by combining a loss function and two regularization terms, aims to find the optimal model parameters that maximize the XGBoost model's performance on the training data while maintaining its generalization ability. The two regularization terms help control the complexity of the XGBoost model and prevent overfitting, while the gradient statistics guide the model's training direction. This design makes XGBoost models perform exceptionally well on large-scale datasets, exhibiting both efficiency and accuracy.

[0075] Step 2: Input the training dataset into the LightGBM model and iteratively train the LightGBM model until one of the following conditions is met: 1. The prediction accuracy of the LightGBM model reaches the preset accuracy threshold. 2. The number of iterations reaches the preset number, specifically: the number of trees reaches the number of trees specified in the target hyperparameter n_estimators of the LightGBM model.

[0076] In step two, the objective function of the LightGBM model is as follows:

[0077]

[0078] Where L(Tm(X)) represents the loss of the LightGBM model on the training dataset X after the m-th iteration; L(Tm-1(X)) represents the loss of the LightGBM model on the training dataset X after the (m-1)-th iteration; J m This represents the set of all leaf nodes in the m-th iteration; λj represents the regularization term for the output value hj at leaf node j, where λj is the regularization parameter used to control overfitting; γ represents the gradient information at leaf node j. j The learning rate of leaf node j, g j (x i ,θ j ) is the gradient of the loss function with respect to the leaf node output θj, I j It is the set of data points on leaf node j.

[0079] In the LightGBM model, g j (x i ,θ jThe loss function (θj) typically represents the first derivative (gradient) of the loss function with respect to the leaf node output θj, and is used to guide the training process of the LightGBM model. By minimizing the loss function, the LightGBM model can learn the inherent patterns in the data and make accurate predictions. The objective function of the LightGBM model describes how to update the loss function in each iteration of training, and optimizes the prediction accuracy of the LightGBM model by introducing regularization terms and gradient information. This optimization strategy makes the LightGBM model efficient and accurate when dealing with large-scale datasets.

[0080] It should be noted that steps one and two above can be executed in any order or simultaneously; there is no restriction on the execution order here.

[0081] It should be noted that the first feature importance extracted from each trained prediction model for N-dimensional equipment features includes the first importance score of each of the N-dimensional equipment features possessed by the training samples input into that prediction model. In other words, the first feature importance is an array of N first importance scores. A higher first importance score indicates a stronger correlation between that equipment feature and the production of defective products, and thus a greater contribution to the prediction model's prediction of defective products during the production process.

[0082] With only a single batch of target products, the sample size is smaller, making it more difficult to accurately analyze the key equipment parameters leading to defects. By combining the LightGBM and XGBoost models, leveraging the stability of XGBoost and the high efficiency of LightGBM for joint training (both are ensemble learning algorithms based on gradient boosting decision trees, capable of evaluating feature importance), the problem is solved. The XGBoost model calculates the average gain of each device feature across all trees; features with higher average gains are considered more important. The LightGBM model uses gradient-based one-side sampling (GOSS) and exclusive feature bundling (EFB) techniques to improve prediction efficiency while also calculating feature importance. Combining the first feature importance extracted from these two decision tree models provides a more comprehensive perspective, as XGBoost is more effective at handling imbalanced data, while LightGBM is more efficient at handling large-scale datasets. These two models evaluate feature importance from different angles to address the feature importance ranking problem when the sample size of a single batch of target products is small but the number of features is large. This allows us to accurately identify the equipment parameters associated with defective products.

[0083] S103: Determine the importance of the second feature of the N-dimensional device feature based on at least two sets of first feature importance.

[0084] In some embodiments, the training dataset is input into the above-mentioned at least two prediction models. After training each prediction model, the first feature importance is extracted from each trained prediction model, that is, at least two sets of first feature importance corresponding one-to-one with the at least two prediction models are obtained. The extracted at least two sets of first feature importance are combined to obtain the second feature importance of N-dimensional equipment features, thereby improving the accuracy of feature importance assessment for defective products generated during the production process of the target product.

[0085] In some embodiments, in order to combine the extracted at least two sets of first feature importance, each trained prediction model is tested on a test dataset to obtain the prediction accuracy of each prediction model after training; weight coefficients are assigned to the at least two prediction models according to the prediction accuracy of each prediction model after training; and a weighted average is performed on the at least two sets of first feature importance according to the weight coefficients of each prediction model to obtain the second feature importance of the N-dimensional device feature.

[0086] It is understandable that the prediction accuracy of a prediction model after training refers to the accuracy of the prediction model in predicting whether the target product is a good product.

[0087] Since different prediction models may use different scales to measure feature importance, in some embodiments, taking a weighted average of at least two sets of first feature importance of the N-dimensional device feature according to the weight coefficient of each prediction model to obtain a second feature importance may include: standardizing each set of first feature importance, and taking a weighted average of the standardized at least two sets of first feature importance according to the weight coefficient of each prediction model to obtain the second feature importance of the N-dimensional device feature.

[0088] The second feature importance includes the second importance score of each of the N-dimensional equipment features. This second importance score represents the degree to which the corresponding equipment feature contributes to the production of defective products during the target product's manufacturing process, after a weighted average of its two first importance scores. Because the second feature importance includes the second importance scores of all N-dimensional equipment features, it achieves quantitative output of feature importance, enhancing the transparency and interpretability of decision-making and enabling non-technical users to understand the decision logic.

[0089] In some embodiments, a linear transformation can be applied to the importance of each group of first features using min-max scaling, so that each first importance score in each group of first feature importance is mapped to a specified range, such as the [0,1] interval. In other embodiments, the importance of each group of first features can be standardized using z-score standardization, transforming the importance of each group of first features based on the mean and standard deviation of the data, so that the transformed data conforms to a standard normal distribution.

[0090] In some embodiments, the higher the prediction accuracy of the prediction model after training, the larger the weight coefficient assigned to it. For example, if the prediction accuracy of the XGBoost model after training is 0.85, and the prediction accuracy of the LightGBM model after training is 0.80, then the weight coefficients of the XGBoost model and the LightGBM model can be assigned weight coefficients of 0.85 and 0.80 respectively, or the weight coefficients of 0.85 and 0.80 can be normalized and assigned.

[0091] Understandably, when weighting at least two groups of first feature importance based on the weight coefficients of each prediction model, the calculation process for the second importance score of each dimension of the device feature in the second feature importance is as follows:

[0092] m i =m 1i ×w 1i +m 2i +w 2i

[0093] Where, m i m is the second importance score of the i-th dimension device feature. 1i w represents the first importance score of the i-th dimension device feature extracted from the XGBoost model. 1i These are the weight coefficients assigned to the XGBoost model, m 2i w represents the first importance score of the i-th dimension device feature extracted from the LightGBM model. 2i These are the weight coefficients assigned to the LightGBM model. The resulting second feature importance includes the second importance scores of N-dimensional equipment features related to the generation of defective products during the production process of the target product: m1, m2, m3, ..., m n .

[0094] For example, the obtained second feature importance includes the second importance score of each dimension of the equipment feature in the N-dimensional equipment feature, as follows: the second importance score of etching time of the etching machine: 0.15, the second importance score of exposure intensity of the lithography machine: 0.12, the second importance score of cleaning solution temperature of the cleaning machine: 0.09, the second importance score of drying temperature of the dryer: 0.07, and the second importance scores of other dimensions of equipment features are all below 0.07.

[0095] S104: Based on the importance of the second feature, determine the cause information of defective products generated during the production process of the target product from the N-dimensional equipment features.

[0096] In some embodiments, based on the second importance score of each dimension of the equipment feature in the second feature importance ranking, K-dimensional equipment features exceeding a preset score threshold are selected from the N-dimensional equipment features, where K is a positive integer less than or equal to N; the cause information of defective products generated during the production process of the target product is located to the K-dimensional equipment features. That is, the K-dimensional equipment features are the cause information of defective products.

[0097] In some embodiments, selecting K-dimensional device features that exceed a preset score threshold from N-dimensional device features based on the second importance score of each device feature in the second feature importance ranking may include: ranking the N-dimensional device features according to their importance scores, for example, ranking the N-dimensional device features from highest to lowest importance based on their second importance scores to obtain a feature ranking list; and obtaining the top K-dimensional device features from the feature ranking list.

[0098] For example, suppose the resulting feature ranking list is as follows: 1. Etching time of the etching machine (second importance score: 0.15); 2. Exposure intensity of the lithography machine (second importance score: 0.12); 3. Cleaning solution temperature of the cleaning machine (second importance score: 0.09); 4. Drying temperature of the dryer (second importance score: 0.07)...

[0099] Taking a preset score threshold of 0.07 as an example, the K-dimensional equipment features with a second importance score higher than the preset score threshold are selected from the feature ranking list. These features are: etching time of the etching machine, exposure intensity of the lithography machine, and cleaning solution temperature of the cleaning machine. Based on the selected K-dimensional equipment features, the reasons for defective products can be determined as follows: 1. Etching time of the etching machine: The etching time may be too long or too short, resulting in uneven etching and affecting the quality of the display panel; 2. Exposure intensity of the lithography machine: Insufficient exposure intensity may lead to unclear pattern transfer, affecting the display effect of the display panel; 3. Cleaning solution temperature of the cleaning machine: The cleaning solution temperature may be too high or too low, resulting in incomplete cleaning and residual impurities affecting the quality of the display panel.

[0100] In other embodiments, a feature sorting list can be directly output, which makes it easier for relevant personnel to identify key equipment parameters that lead to defective products.

[0101] Since different prediction models have varying sensitivities to different types and distributions of data, the hybrid strategy based on one or more of the above embodiments can enhance the robustness of the model. Combining the importance of first features extracted from multiple prediction models can reduce the potential bias in the training results of individual prediction models and improve the accuracy of feature importance assessment.

[0102] The selected K-dimensional equipment features provide a clear direction and basis for quality improvement in the production process of the target product. In some embodiments, in order to more accurately locate the cause information of defective products, the cause information of defective products in the production process of the target product is determined from the K-dimensional equipment features according to the importance of the second feature. This may further include: obtaining historical data of the process equipment corresponding to the K-dimensional equipment features; and determining the cause information of defective products in the production process of the target product from the K-dimensional equipment features based on the historical data, thereby enabling more accurate location of the cause of defective products.

[0103] In some embodiments, historical data includes the operation logs and maintenance records of the process equipment. Taking the characteristics of K-dimensional equipment, including the etching time of the etching machine, the exposure intensity of the lithography machine, and the cleaning solution temperature of the cleaning machine, as an example, the operation logs and maintenance records of each process equipment in the etching machine, lithography machine, and cleaning machine can be obtained: If the operation log of the etching machine indicates that the recent etching time fluctuates and coincides with the time period when defective products are generated, it is further determined that the poor etching time of the lithography machine is causing defective products; if the maintenance record of the lithography machine shows that the light source of the lithography machine was recently replaced, it is further determined that the reason for the defective products is that the performance of the new light source is unstable, resulting in abnormal exposure intensity. According to the operation log of the cleaning machine, the temperature change curves of the cleaning machine during the production process of M batches of target products are obtained, and the temperature change curves are analyzed to see if there are abnormal fluctuations or exceed the normal range. If so, it is determined that the sensor used to detect the cleaning solution temperature malfunctioned for a certain period of time, resulting in inaccurate temperature control, and it is further determined that the temperature sensor of the cleaning machine is abnormal, causing defective products.

[0104] In some embodiments, after determining the cause information of defective products generated during the production process of the target product, the process may further include: improving the operating parameters or hardware of the process equipment corresponding to the K-dimensional equipment features based on the cause information, so as to eliminate the factors that cause defective products during the production process of the target product. For example: adjusting the etching time of the etching machine to ensure that the etching time of the etching machine is within the optimal range; or adjusting and optimizing the light source equipment of the lithography machine to ensure that the exposure intensity is stable within the preset range; or repairing the temperature sensor of the cleaning machine to optimize the temperature control parameters of the cleaning solution and improve the yield of the target product in subsequent batches.

[0105] In some embodiments, after improving the operating parameters or hardware of the process equipment corresponding to the K-dimensional equipment features, production process data and product defect rate are continuously monitored to verify the effectiveness of the improvements. If the defect rate is significantly reduced, it indicates that the improvement measures are effective.

[0106] Figure 2 It shows Figure 1 An exemplary process for determining the cause of a defect in a process. To facilitate understanding of the technical solution provided by this invention, reference is made below. Figure 2 An exemplary flow of the method for determining the cause of a defect provided in this embodiment of the invention is given as follows:

[0107] Step S201: Input the training dataset into the initialized XGBoost model and train the XGBoost model;

[0108] Step S202: Extract the first set of first feature importance from the trained XGBoost model. The first set of first feature importance includes the first importance score of the N-dimensional device features.

[0109] Step S203: Input the training dataset into the initialized LightGBM model and train the LightGBM model;

[0110] Step S204: Extract the second set of first feature importance from the trained LightGBM model. The second set of first feature importance includes the first importance score of the N-dimensional device features.

[0111] Step S205: Assign weight coefficients based on the prediction performance of the trained XGBoost model and the trained LightGBM model.

[0112] Step S206: Calculate the second feature importance by weighted averaging the two sets of first feature importance and the weight coefficients assigned to the XGBoost model and the LightGBM model. The second feature importance includes the second importance score of the N-dimensional device features.

[0113] Step S207: Sort the features according to their importance based on the importance of the second feature, and obtain a ranking of the features from high to low importance;

[0114] Step S208: Output the ranking of feature importance.

[0115] Through the above steps S201 to S208, the training dataset related to the generation of defective products is input into the XGBoost model and the LightGBM model for training, and the feature importance extracted from the XGBoost model and the LightGBM model after training is combined to rank the feature importance, thereby more accurately analyzing the equipment parameters that cause defective products.

[0116] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, such as... Figure 3 As shown, the electronic device includes a memory 304, a processor 302, and a computer program stored in the memory 304 and executable on the processor 302, wherein the processor 302 executes the program using the method described above for determining the cause of a malfunction.

[0117] Among them, Figure 3 In this document, a bus architecture (represented by bus 300) is used. Bus 300 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 302 and memory represented by memory 304. Bus 300 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 305 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 302 is responsible for managing bus 300 and general processing, while memory 304 can be used to store data used by processor 302 during operation.

[0118] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for enabling the computer to determine the cause of product manufacturing defects.

[0119] It should be noted that the accompanying drawings of the embodiments of the present invention only involve structures relevant to the embodiments of the present invention; other structures can be referred to with conventional designs. Where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other to obtain new embodiments.

[0120] Although some embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make further changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

Claims

1. A method for determining the cause of a defect, characterized in that, include: Obtain equipment feature data of the production process of the target product, wherein the equipment feature data includes feature values ​​of N-dimensional equipment features of the process equipment used to produce the target product, where N is an integer greater than 1; The equipment feature data is used to train at least two prediction models, and the first feature importance of the N-dimensional equipment features is extracted from each of the trained prediction models to obtain at least two sets of first feature importance. Each set of first feature importance represents the degree of contribution of the N-dimensional equipment features to the production of defective products in the target product. The importance of the second feature of the N-dimensional device feature is determined based on the importance of the at least two sets of first features; Based on the importance of the second feature, the cause information of defective products generated during the production process of the target product is determined from the N-dimensional equipment features.

2. The method as described in claim 1, characterized in that, The acquisition of equipment characteristic data of the target product's production process includes: Obtain production process data for M batches of the target product, where M is a positive integer; The production process data of the M batches are filtered in multiple stages to obtain the equipment feature data. The equipment feature data includes multiple training samples. Each training sample has feature values ​​of various dimensions of equipment features related to the production of defective products when producing one of the target products.

3. The method as described in claim 2, characterized in that, The acquisition of production process data for M batches of the target product includes: Throughout the entire production process of the target product, all production process data is recorded and stored in a database. The full production process data includes the parameter values ​​of the equipment parameters of the process equipment used when producing each of the target products. After M batches of the target product have been produced, obtain the defect rate of the M batches of the target product; If the defect rate of the target product in M ​​batches is greater than a preset threshold, import the production process data of the target product in M ​​batches from the database.

4. The method as described in claim 2, characterized in that, The equipment characteristic data is obtained by performing multi-level filtering on the production process data of the M batches, including: The production process data is preprocessed to perform a first-level data filtering to obtain a first dataset. The first dataset is filtered through feature engineering to obtain the second dataset. The second dataset is input into a pre-trained anomaly detection model, which then filters out anomalous samples from the second dataset to obtain the device feature data.

5. The method as described in claim 1, characterized in that, The step of training at least two prediction models using the device feature data includes: The equipment feature data is subjected to equipment feature selection to obtain a training dataset. The training dataset includes multiple training samples. Each training sample has multi-dimensional equipment features. The correlation between the multi-dimensional equipment features and the production of defective products is higher than the correlation between the various dimensions of equipment features that are removed during equipment feature selection and the production of defective products. The N-dimensional equipment features belong to the multi-dimensional equipment features. The training dataset is input into the at least two prediction models to train the at least two prediction models. During the training process of each prediction model, the prediction model is used to predict whether a corresponding target product is a good product based on the training samples.

6. The method as described in claim 5, characterized in that, The training samples input to the at least two prediction models are the same.

7. The method as described in claim 5, characterized in that, Before inputting the training dataset into the at least two prediction models and training the at least two prediction models, the method further includes: Determine the target hyperparameters for each of the at least two prediction models; For each prediction model, the prediction model is initialized according to the target hyperparameters of the prediction model.

8. The method as described in claim 5, characterized in that, The at least two prediction models include a first decision tree model and a second decision tree model; The step of inputting the training dataset into the at least two prediction models and training the at least two prediction models includes: The training dataset is input into the first decision tree model for training, and the training dataset is input into the second decision tree model for training. A set of first feature importance is extracted from the trained first decision tree model, and another set of first feature importance is extracted from the trained second decision tree model. The model structure of the first decision tree model is different from that of the second decision tree model. Each group of first feature importance includes the first importance score of the N-dimensional device feature. The first importance score represents the degree of contribution of the corresponding dimensional device feature to the prediction result. The prediction result is the result of predicting whether the target product is a good product.

9. The method according to any one of claims 1-8, characterized in that, The step of determining the second feature importance of the N-dimensional device features based on the at least two sets of first feature importance includes: Each of the trained prediction models is tested on a test dataset to obtain the prediction accuracy of each of the trained prediction models. Based on the prediction accuracy of each prediction model after training, weight coefficients are assigned to the at least two prediction models respectively; The importance of the second feature is obtained by weighting the importance coefficients of the at least two groups of first features according to the weight coefficients of each prediction model.

10. The method according to any one of claims 1-8, characterized in that, The second feature importance includes the second importance score of each dimension of the equipment feature in the N-dimensional equipment features. The second importance score corresponding to each dimension of the equipment feature represents the degree of contribution of the equipment feature in that dimension to the production of defective products in the target product after weighted averaging. The step of determining the cause information of defective products generated during the production process of the target product from the N-dimensional equipment features based on the importance of the second feature includes: The step involves selecting K-dimensional device features from the N-dimensional device features based on the second importance score of each device feature in the second feature importance, where K is a positive integer less than or equal to N. The cause information of defective products generated during the production process of the target product is located to the K-dimensional equipment feature.

11. The method as described in claim 10, characterized in that, The step of determining the cause information of defective products generated during the production process of the target product from the N-dimensional equipment features based on the importance of the second feature further includes: Obtain historical data of the process equipment corresponding to the K-dimensional equipment features; Based on the historical data, the reasons for the defects in the production process of the target product are determined from the K-dimensional equipment features.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for determining the cause of a malfunction as described in any one of claims 1 to 11.

13. A computer-readable storage medium storing computer instructions for causing the computer to perform the method for determining the cause of a malfunction as described in any one of claims 1 to 11.

Images