Multi-process manufacturing quality prediction method based on autoencoder and graph attention network

Abstract

The present invention relates to a multi-process manufacturing quality prediction method based on an autoencoder and a graph attention network. The method includes: obtaining the time series feature data obtained by the process sensor detection of each process on the production line, constructing a total process data set, inputting it into a semiconductor multi-process quality prediction model, preprocessing the data set, grouping the features according to the number of stages, and inputting the features in each stage into a multi-layer stacked autoencoder, reducing the features to the same dimension to obtain the final intermediate features, and using the intermediate features as the node input of the graph attention network, iteratively updating to obtain the final node output, wherein the graph attention network adopts a residual mechanism, and uses the final node output as the input of the final feedforward neural network to obtain quality feature regression prediction. The present invention realizes multi-dimensional and multi-stage modeling by combining autoencoder feature dimensionality reduction technology with graph neural network data mining technology.

Description

Technical Field

[0001] The present invention relates to a multi-process manufacturing quality prediction method based on an autoencoder and a graph attention network, and belongs to the technical field of quality prediction. Background Art

[0002] Semiconductor chip manufacturing is a complex process involving coupling and complex associations between multiple processes. The quality changes of products in the wafer processing, unit assembly, system packaging and other stages will be transmitted to each other, and the error of the final product quality is the accumulation of errors in each stage. The key lies in continuous process optimization to improve the yield, which is mainly reflected in the complexity of the process and the complexity of the equipment. Semiconductor manufacturing involves a variety of precise physical and chemical processes and more than a hundred processes. Each process requires highly precise control. At the same time, the parameters of production equipment such as thin film deposition and lithography have a direct impact on product quality. In order to improve the overall quality, multi-process quality prediction is an important management method. By predicting and monitoring the quality status of each process, potential problems can be discovered early and defects in subsequent processes can be prevented. However, the huge amount of data in the process needs to be effectively collected, integrated and processed, and models need to be built to utilize this data. This is the current key challenge.

[0003] The existing multi-process quality prediction methods mainly include the following: First, based on statistical methods: It usually assumes independence and identical distribution between data. However, in the actual multi-process manufacturing process, the data between each process may have complex correlations and non-linear relationships. The limitations of this assumption will cause the model to be unable to capture the potential mutual influence and complex dynamic changes between processes, reducing the applicability and prediction ability of the prediction model. Second, prediction based on physical models: Based on the manufacturing process and physical properties of semiconductor wafers, a semiconductor wafer physical model is established to predict wafer quality. However, hundreds of semiconductor processes and months of manufacturing cycles make the modeling process more complicated. Third, prediction based on machine learning methods: Introducing machine learning technology into the field of product quality prediction, constructing regression or classification models to predict product quality levels can improve prediction accuracy and stability, but requires more data and computing resources.

[0004] The manufacturing process of multi-process products has a complex mechanism, and there are a large number of complex nonlinear relationships between various quality characteristics and product quality. Traditional statistical methods have problems such as low accuracy and poor robustness when facing quality prediction of complex products. Traditional machine learning methods rely heavily on manual feature engineering processes, which requires experts to spend a lot of time and energy to design features or feature preprocessing methods before building traditional machine learning models, which weakens the robustness and portability of traditional machine learning methods. At present, the research on quality prediction methods based on machine learning is mainly focused on a single process, while there is less exploration of quality prediction in multi-stage manufacturing processes (MMPs). Because MMP does not model the dependencies between each stage, it cannot handle more complex stage dependencies, nor can it predict the measurement results of different stages. Summary of the invention

[0005] The present invention provides a multi-process manufacturing quality prediction method based on an autoencoder and a graph attention network, aiming to solve at least one of the technical problems existing in the prior art.

[0006] The technical solution of the present invention relates to a multi-process manufacturing quality prediction method based on an autoencoder and a graph attention network. The method according to the present invention comprises the following steps:

[0007] S100, obtaining time series feature data obtained by process sensor detection of each process on the production line, constructing a total process data set, inputting it into a semiconductor multi-process quality prediction model, and preprocessing the data set;

[0008] S200, after grouping the features according to the number of stages, the features in each stage are input into a multi-layer stacked autoencoder to reduce the features to the same dimension to obtain the final intermediate features;

[0009] S300, using the intermediate features obtained as node inputs of a graph attention network, and iteratively updating to obtain final node outputs; wherein the graph attention network adopts a residual mechanism;

[0010] S400, using the final node output as the input of the final feedforward neural network to obtain quality feature regression prediction.

[0011] Furthermore, in step S100, the semiconductor multi-process quality prediction model is trained using the total process data set.

[0012] Further, for step S200, the multi-layer stacked autoencoder includes an encoder that compresses input data into a latent space representation and a decoder that compresses input data into a latent space representation.

[0013] Further, the step S200 includes:

[0014] S210, the encoder converts the high-dimensional input into Compressed into low-dimensional hidden feature representation , the formula is as follows:

[0015]

[0016] In the formula, and represents the weight matrix and bias in the encoder; and Represent the vectors of the input layer and hidden layer respectively;

[0017] in, , A parameter set used to represent the encoder;

[0018] S220, the decoder represents the hidden feature through the following mapping function Reconstruct back to the reconstructed input , the formula is as follows:

[0019]

[0020] In the formula, and represents the weight matrix and bias in the decoder, is the corresponding activation function; where , The set of parameters used to represent the weight matrix and bias in the decoder;

[0021] S230, As the input of the next encoder layer , and so on until the last layer of encoder-decoder structure, take the last layer As the feature vector after dimensionality reduction in this process stage.

[0022] Further, in step S200:

[0023] To obtain the optimal network parameters, the BP algorithm is used to minimize the following reconstruction error loss function:

[0024]

[0025] In the formula, N represents the number of training samples; and denote the weight matrix and bias in the encoder respectively, , Represent the weight matrix and bias in the decoder respectively; Represents the input of the n-th layer encoder structure; Represents the output of the n-th layer decoder structure; Indicates encoder input To hidden layer features The mapping relationship; express To decoder output The mapping relationship.

[0026] Further, in step S300:

[0027] In the graph attention network with a residual mechanism in each layer, the input of the next layer is the sum of the output of the previous layer and the input of the previous layer.

[0028] Further, in step S400:

[0029] Obtain the adjacency matrix based on the actual production process and the dependencies between the manufacturing stages on the production line , the forward GAT of the lth layer is used to encode the structural information of the machine dependency graph, which is formulated as follows:

[0030]

[0031] In the formula, Represents the attention score of nodes a and b; Represents the normalized attention score of nodes a and b; represents the attention score of nodes a and c; Represents the feature vector representation of node a in the l+1 layer graph network; Represents the feature vector representation of node b in the l-layer graph network; , , is a learnable parameter, and LeakyReLU is the activation function; Indicates the neighboring or related manufacturing node stage of stage a; superscript Indicates Layer Graph Attention Networks.

[0032] Further, in step S500:

[0033] When obtaining the quality feature prediction of the sth stage, the last layer of the graph attention network can be Connect the last feedforward neural network to complete the quality feature regression prediction, and the formula is as follows:

[0034]

[0035] In the formula, represents the prediction result of the quality characteristics of the sth stage; Represents the feature vector of the corresponding node in the s stage in the last layer L of the graph attention network;

[0036] Among them, the following mean square error loss function is minimized through multiple rounds of iterative BP algorithm to obtain the optimal network parameters:

[0037]

[0038] In the formula, The true value of the quality characteristic; Represents the predicted value of quality characteristics.

[0039] The technical solution of the present invention also relates to a computer-readable storage medium on which program instructions are stored, and the above-mentioned method is implemented when the program instructions are executed by a processor.

[0040] The technical solution of the present invention also relates to a multi-process manufacturing quality prediction system based on an autoencoder and a graph attention network, wherein the system includes a computer device comprising the above-mentioned computer-readable storage medium.

[0041] The beneficial effects of the present invention are as follows:

[0042] The present invention is based on a multi-process manufacturing quality prediction method and system based on an autoencoder and a graph attention network, and solves the technical problems of dimensionality disaster and multi-stage modeling through a multi-process manufacturing quality prediction method based on the combination of autoencoder feature dimensionality reduction technology and graph neural network data mining technology. The multi-layer stacked autoencoder (SAE, Stacked Auto-Encoder) used in the present invention realizes the extraction of high-dimensional features, which is used to solve the feature engineering problem of high-dimensional data disaster. In addition, a graph attention network (GAT, Graph Attention Networks) framework is proposed, and the dependency relationship between different stages is modeled using the adjacency matrix, which further realizes data mining and feature fusion, and finally combines the attention mechanism to realize the regression prediction of the final quality target. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a basic flow chart of the multi-process manufacturing quality prediction method according to the present invention.

[0044] Figure 2 Schematic diagram of the structure of a multi-layer stacked autoencoder according to the method of the present invention.

[0045] Figure 3 Schematic diagram of the structure of a single-layer autoencoder according to the method of the present invention.

[0046] Figure 4 Schematic diagram of the structure of the graph attention network according to the method of the present invention. DETAILED DESCRIPTION

[0047] The concept, specific structure and technical effects of the present invention will be clearly and completely described below in combination with the embodiments and drawings to fully understand the purpose, scheme and effect of the present invention.

[0048] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to another feature, or it may be indirectly fixed or connected to another feature. The singular forms of "," "said" and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art. The terms used in this specification are intended to describe specific embodiments only and are not intended to limit the invention. The term "and / or" used herein includes any combination of one or more of the related listed items.

[0049] It should be understood that, although the term first, second, third etc. may be adopted to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish the same type of elements from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as" etc.) provided herein is only intended to better illustrate embodiments of the present invention, and unless otherwise required, will not impose limitations on the scope of the present invention.

[0050] Reference Figures 1 to 4 In some embodiments, the multi-process manufacturing quality prediction method based on autoencoder and graph attention network according to the present invention comprises at least the following steps:

[0051] S100, obtaining the time series feature data obtained by the process sensor detection of each process on the production line, constructing a total process data set, inputting it into the semiconductor multi-process quality prediction model, and preprocessing the data set. The semiconductor multi-process quality prediction model is trained with the total process data set, wherein the preprocessed data set is divided into a training set and a test set according to a certain ratio.

[0052] S200, after grouping the features according to the number of stages, the features in each stage are input into a multi-layer stacked autoencoder to reduce the features to the same dimension to obtain the final intermediate features.

[0053] S300, obtaining the intermediate features as node inputs of a graph attention network, and iteratively updating to obtain final node outputs; wherein the graph attention network adopts a residual mechanism.

[0054] S400, using the final node output as the input of the final feedforward neural network to obtain quality feature regression prediction.

[0055] Furthermore, the present invention optimizes network parameters by minimizing the mean square error loss function through multiple rounds of iterative back propagation process and gradient descent algorithm. The present invention uses the validation set to enter the model to evaluate the model prediction ability. This process does not update the model parameters, but is used to adjust the model hyperparameters, such as the number of training rounds, learning rate, optimizer selection and parameter settings. When the model accuracy meets the industrial generation requirements, the model can be put online to achieve quality prediction and supervision of new samples.

[0056] The present invention realizes high-dimensional nonlinear dimensionality reduction through the SAE module, and proposes a reasonable quality prediction model in combination with the graph neural network, which reflects the quality propagation and the correlation between multiple quality indicators and stages, so as to obtain the transmission relationship between the feature information of each stage. A high-accuracy cascade quality prediction model is built. The whole process is an end-to-end model, which alleviates the problem that other methods rely heavily on the manual feature engineering process and require experts to spend a lot of time and energy to design features or feature preprocessing, greatly improving the robustness of the model.

[0057] The present invention constructs a training set by acquiring the process parameters of each process in the historical product processing and the time series feature data of multi-source sensors. Before modeling, the data set needs to be preprocessed, mainly including useless feature deletion, collinear feature deletion, category coding, missing value filling, data standardization and sample division. It should be noted that other existing data preprocessing methods can also be used.

[0058] After completing data preprocessing, the preprocessed data set is divided into a training set and a test set according to a certain ratio. For samples in the training data set, the features are grouped according to the S stage. , Specifically, let the number of features contained in each stage after grouping be .

[0059] Then set the model hyperparameters. Figure 2, initialize the trainable parameters of SAE and GAT. It should be noted that the autoencoder (AE) is an unsupervised neural network and a generative model. The core idea is to learn the effective encoding of data, with the aim of reconstructing the original input as accurately as possible in the output. It consists of an encoder and a decoder. The encoder compresses the input data into a representation of the latent space, and the decoder restores this representation to an approximation of the original data. In this way, the autoencoder can learn the inherent laws and structures of the input data. In addition, in the autoencoder, the input data is compressed into a low-dimensional vector by the encoder, which is called a "latent representation" or "hidden representation". The decoder decodes this low-dimensional vector into a vector with the same dimension as the input data. The whole process can be regarded as a process of encoding and decoding the input data.

[0060] Specifically, see Figure 3 , the vectors of the input layer and the hidden layer can be used and Represents that the reconstructed input vector of the output layer can be expressed as The encoder (AE) is responsible for converting high-dimensional input Compressed into low-dimensional hidden feature representation , the formula is as follows:

[0061]

[0062] In the formula, and represents the weight matrix and bias in the encoder; where, , Used to represent the parameter set of the encoder.

[0063] Then, the decoder transforms the hidden feature representation into Reconstruct back to the reconstructed input , the formula is as follows:

[0064]

[0065] In the formula, and represents the weight matrix and bias in the decoder, is the corresponding activation function; where , The set of parameters used to represent the weight matrix and bias in the decoder.

[0066] Then, As the input of the second layer AE (next AE), and so on to form a stacked autoencoder. As the input of the next encoder layer , and so on until the last layer of encoder-decoder structure, take the last layer As the feature vector after dimensionality reduction in this process stage.

[0067] To obtain the optimal network parameters, the BP algorithm is used to minimize the following reconstruction error loss function:

[0068]

[0069] In the formula, N represents the number of training samples; and denote the weight matrix and bias in the encoder respectively, , Represent the weight matrix and bias in the decoder respectively; Represents the input of the n-th layer encoder structure; Represents the output of the n-th layer decoder structure; Indicates encoder input To hidden layer features The mapping relationship; express To decoder output The mapping relationship.

[0070] The characteristics in each stage The features are reduced to the same dimension through different multi-layer stacking autoencoders Get the final intermediate features , which is used as the node input of the subsequent graph attention network. As the input of the residual-based graph attention network (GAT), iterative updates are performed to obtain the final node output , where the superscript l represents the last l-th layer of GAT (where l here is a letter, not the number 1).

[0071] The graph attention network of the present invention has L layers of GAT, and each layer is combined with a residual mechanism, that is, the input of the next layer is the sum of the output of the previous layer and the input of the previous layer. The input feature of the lth layer is the updated node feature of the (l - 1)th layer plus the original input of the (l - 1)th layer. Figure 4 , the directed solid and dotted lines in the figure represent the adjacency relationship, that is, which nodes the feature of this node (stage) needs to be fused with. It is designed according to the actual production process on the production line, as shown in Figure 4 The corresponding adjacency matrix is ​​as follows:

[0072]

[0073] In the formula, It means that the jth node points to the i-th node. When the i-th node feature is updated, the information of the j-th node needs to be integrated.

[0074] Specifically, given the adjacency matrix designed according to the manufacturing stage dependencies , where the forward GAT of the lth layer is used to represent the structural information of the encoding machine dependency graph, which can be formulated as follows:

[0075]

[0076] In the formula, Represents the attention score of nodes a and b; Represents the normalized attention score of nodes a and b; Represents the attention score of nodes a and c; Represents the feature vector representation of node a in the l+1 layer graph network; Represents the feature vector representation of node b in the l-layer graph network; , , is a learnable parameter, and LeakyReLU is the activation function; Indicates the neighboring or related manufacturing node stage of stage a; superscript Indicates the last Layer graph attention network; the graph attention network of the lth layer is used to represent the structural information of the encoded machine dependency graph.

[0077] Furthermore, in order to stabilize the learning process, the present invention adopts a multi-head attention mechanism. The output of the GAT node in the last layer is It represents the final characteristics of each stage. For example, when obtaining the quality feature prediction of the sth stage, Connect the last feedforward neural network to complete the quality feature regression prediction, and the formula is as follows:

[0078]

[0079] In the formula, represents the prediction result of the quality characteristics of the sth stage; Represents the feature vector of the corresponding node in the last layer L of the graph attention network at stage s.

[0080] In order to obtain the optimal network parameters, the following mean square error loss function is minimized through multiple rounds of iterative BP algorithm:

[0081]

[0082] In the formula, The true value of the quality characteristic; represents the predicted value of the quality feature. The remaining hyperparameters are optimized by grid search.

[0083] It should be appreciated that the method steps in the embodiments of the present invention can be implemented or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level process or object-oriented programming language to communicate with a computer system. However, if necessary, the program can be implemented in an assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed ASIC for this purpose.

[0084] In addition, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0085] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, an RSM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention can also include the computer itself.

[0086] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on the display.

[0087] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical scheme and / or implementation method may have various modifications and changes.

Claims

1. A multi-process manufacturing quality prediction method, characterized in that: The method comprises the following steps: S100, obtaining time series feature data detected by process sensors of each process on the production line, constructing a total process data set, inputting it into a semiconductor multi-process quality prediction model, and preprocessing the data set; wherein the semiconductor multi-process quality prediction model is trained by the total process data set; S200, after grouping the features according to the number of stages, the features in each stage are input into a multi-layer stacked autoencoder to reduce the features to the same dimension to obtain the final intermediate features; S300, using the intermediate features obtained as node inputs of a graph attention network, and iteratively updating to obtain final node outputs; wherein the graph attention network adopts a residual mechanism; S400, using the final node output as the input of the final feedforward neural network to obtain quality feature regression prediction; Wherein, in step S400: According to the actual production process on the production line, given the adjacency matrix A obtained according to the manufacturing stage dependency i , among which The forward GAT of the layer is used to represent the structural information of the encoding machine dependency graph, and the formula is as follows: ， In the formula, Represents the attention score of nodes a and b; Represents the normalized attention score of nodes a and b; Represents the attention score of nodes a and c; Represents the feature vector of node a in the l+1 layer graph network; express Feature vector representation of node b in the layer graph network; , , is a learnable parameter, and LeakyReLU is the activation function; Indicates the neighboring or related manufacturing node stage of stage a; superscript Indicates Layer Graph Attention Network; Among them, when obtaining the quality feature prediction of the sth stage, the last layer of the graph attention network can be Connect the last feedforward neural network to complete the quality feature regression prediction, and the formula is as follows: ； In the formula, represents the prediction result of the quality characteristics of the sth stage; , Represent the weight matrix and bias in the decoder respectively; Represents the feature vector of the node corresponding to stage s in the last layer L of the graph attention network; Among them, the following mean square error loss function is minimized through multiple rounds of iterative BP algorithm to obtain the optimal network parameters: ； In the formula, The true value of the quality characteristic.

2. The method according to claim 1, characterized in that For step S200, the multi-layer stacked autoencoder includes an encoder that compresses input data into a latent space representation and a decoder that compresses input data into a latent space representation.

3. The method according to claim 2, characterized in that Regarding step S200, step S200 includes: S210, the encoder compresses the high-dimensional input into a low-dimensional hidden feature representation through a nonlinear activation function , the formula is as follows: ， In the formula, and represents the weight matrix and bias in the encoder; A vector representing the input layer; represents hidden feature representation; in, , A parameter set used to represent the encoder; S220, the decoder represents the hidden feature through the following mapping function Reconstruct back to the reconstructed input , the formula is as follows: ， In the formula, and represents the weight matrix and bias in the decoder, is the corresponding activation function; where , The set of parameters used to represent the weight matrix and bias in the decoder; S230, the hidden layer features As the input of the next encoder layer , and so on until the last layer of encoder-decoder structure, take the last layer As the feature vector after dimensionality reduction in this process stage.

4. The method according to claim 3, characterized in that In step S200: To obtain the optimal network parameters, the BP algorithm is used to minimize the following reconstruction error loss function: ， In the formula, N represents the number of training samples; and Represent the weight matrix and bias in the encoder respectively; Represents the input of the n-th layer encoder structure; Represents the output of the n-th layer decoder structure; Indicates encoder input To hidden layer features The mapping relationship of express To decoder output The mapping relationship.

5. The method according to claim 4, characterized in that In step S300: In the graph attention network with a residual mechanism in each layer, the input of the next layer is the sum of the output of the previous layer and the input of the previous layer. 6 . A computer-readable storage medium having program instructions stored thereon, wherein the program instructions are executed by a processor to implement the method according to claim 1 .

7. A multi-process manufacturing quality prediction system, characterized in that: include: A computer device comprising a computer readable storage medium according to claim 6.

Images