Multi-process manufacturing process quality monitoring method and device and storage medium

Abstract

The invention relates to a multi-process manufacturing process quality monitoring method and device and a storage medium, and the method comprises the steps: obtaining multi-modal data and quality data from a multi-process manufacturing process based on an intelligent sensor and a data collection technology; preprocessing the multi-modal data; determining a manufacturing process which has the greatest influence on the quality of a final product, and establishing a quality check point in a key process; establishing and training an artificial neural network auto-encoder model, and performing fusion and feature extraction on multi-modal data; single-process, cross-process and multi-process online monitoring systems are respectively established; and performing diagnosis and root cause analysis on abnormal problems found in monitoring, and updating a detection strategy and a model as required. According to the characteristics of the multi-process manufacturing process, a comprehensive online monitoring system is established, the quality process management and control of the production process are guided online in real time, abnormal conditions are diagnosed, and the problem processing and decision-making capabilities are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing and quality control, and in particular to a multi-step manufacturing process quality monitoring method, device and storage medium. Background Art

[0002] Product quality is an important factor in evaluating production capacity and improving market competitiveness, and is the focus of enterprises. Statistical process control (SPC), as a process control tool using mathematical statistics, can analyze and pre-control the production process, so that the process is maintained in a controlled state affected only by random factors, so as to achieve the purpose of controlling quality, improving quality, and continuous improvement. Traditional SPC methods, such as control charts, have been widely used for monitoring quality data due to their applicability and simplicity, and are mainly used in low-complexity and stable manufacturing processes.

[0003] With the presence of high-dimensional variables, dynamic and uncertain environmental factors, manufacturing and production processes have become more complex. Complex industrial processes usually consist of multiple manufacturing processes. For example, the processing of semiconductor products has hundreds of processes, including deposition, lithography, plasma etching, ion implantation, chemical mechanical polishing, etc. As the most common complex production method in the manufacturing industry, multi-process manufacturing has its typical characteristics, that is, the processes affect each other and error fluctuations are transmitted, which also makes the quality control of multi-process manufacturing processes more difficult. Traditional SPC technology has become inflexible in dealing with current problems and challenges. In a multistage manufacturing process (MMS), many factors (such as equipment and manufacturing variables) have interactive and cumulative effects on the quality of the final product, and there is a correlation between different processes. Problems in the previous process will have a certain impact on the subsequent ones. Reference Figure 2 Different processes also have the characteristics of transmission. Once a quality error occurs, the error will be transmitted from the upstream process to the downstream process as the processing proceeds.

[0004] For each process, the error comes from two aspects: the state of the current process and the error fluctuation caused by the previous process. The source of quality error is complex in the context of multiple processes. Due to the complex connections between processes, the actual factors affecting product quality are difficult to capture. Faced with the coupling characteristics of different processes in complex production processes, the real source of quality fluctuations often becomes more complicated. Therefore, in the face of the complex characteristics unique to multi-process manufacturing processes, it is necessary to model and analyze the manufacturing process and develop a reasonable online monitoring method to ensure product quality.

[0005] The development of sensor technology enables the automatic collection and recording of equipment data and test data in the manufacturing process. These data provide information reflecting the current state of the system and make early detection of faults possible. However, in modern industry, the data structure is becoming more and more complex, making it difficult to detect faults in the entire production system. For example, in different processes of semiconductor manufacturing, the corresponding output can be image types (such as film thickness at different positions on the wafer, images of etched grooves captured by scanning electron microscopes), and various function curves (such as temperature, pressure and radio frequency curves during the reaction process). These multimodal data reveal the state characteristics of the system from different angles, increase the observation range of the system in time and space, and also pose challenges to the modeling, feature extraction and quality control of multi-process manufacturing data. At the same time, due to poor calibration and measurement errors or various other reasons, the collected data usually has missing values, and the sample size is unbalanced due to the low frequency of some faults. Therefore, how to comprehensively utilize multidimensional, multimodal, and incomplete data to perform statistical modeling and online monitoring of multi-process manufacturing processes is an urgent problem to be solved in modern industry.

[0006] Existing quality monitoring methods mostly focus on the problems of a single mode and a single process. However, in the case of complex data structures with multiple processes, conventional methods lose the relationship between processes and the correlation of multimodal data. They do not consider the impact of the mixed causes of complex data. There is a lack of a complete system for comprehensive monitoring of the manufacturing process, and it is impossible to comprehensively monitor multi-process industrial manufacturing processes. Summary of the invention

[0007] The present invention provides a multi-step manufacturing process quality monitoring method, device and storage medium, aiming to solve at least one of the technical problems existing in the prior art.

[0008] The technical solution of the present invention is a multi-process manufacturing process quality monitoring method, which is applied to a multi-process manufacturing process quality monitoring device, which includes a data acquisition unit, which is based on intelligent sensors and data collection technology to obtain multimodal data and quality data from the multi-process manufacturing process; a data processing unit, which preprocesses the multimodal data and quality data and establishes quality checkpoints, and the data processing unit is connected to the data acquisition unit; a model training unit, which establishes and trains an artificial neural network autoencoder (Autoencoders) model, and the model training unit is connected to the data processing unit; a detection unit, which performs online monitoring on a single process, cross-process and multiple processes and outputs diagnosis results and root cause analysis, and the detection unit is connected to the data processing unit. The multi-process manufacturing process quality monitoring method includes the following steps:

[0009] S100, based on intelligent sensors and data collection technology, obtains multimodal data and quality data from multi-process manufacturing processes;

[0010] S200, preprocessing the multimodal data to process outliers and missing values ​​in the multimodal data;

[0011] S300, determine the manufacturing process that has the greatest impact on the quality of the final product and establish quality checkpoints in key processes;

[0012] S400, establish and train artificial neural network autoencoder (Autoencoders) model to fuse and extract features of multimodal data;

[0013] S500, based on the artificial neural network autoencoder (Autoencoders) model, establish single-process, cross-process and multi-process online monitoring systems respectively;

[0014] S600: diagnose and analyze the root causes of abnormal problems found during monitoring, and update the detection strategy and artificial neural network autoencoder (Autoencoders) model as needed.

[0015] Further, in step S100, the multimodal data includes at least maintenance records, state variables, process parameters, intermediate product characteristics, final product quality, and environmental sensor variables;

[0016] Determine quality data based on expert knowledge and previous manufacturing experiments, use different collection methods according to the data interface of different collection objects, and record quality data in real time;

[0017] In step S200, for abnormal values ​​of multimodal data, by setting a threshold and interval, abnormal data points beyond the interval are judged and eliminated; for local missing data caused by a temporary failure of the detection equipment, interpolation processing is performed; for data with different dimensions, data with a unified scale are standardized;

[0018] For missing values ​​of multimodal data, the missing data are processed by interpolation. The interpolation methods include at least single interpolation methods such as deletion, mean, random regression interpolation and regression interpolation, as well as multiple interpolation methods such as Markov chain Monte Carlo (MCMC) method and expectation maximization (EM) algorithm to interpolate missing values.

[0019] Further, the step S300 includes:

[0020] S310, determining possible factors as target variables;

[0021] S320, screen key paths, variables, equipment, processes and parameters through evaluation and adaptability testing;

[0022] S330. Sort the candidate causes of the abnormality, screen the faults in the manufacturing system one by one, and take measures on the equipment as soon as possible.

[0023] Further, in step S400,

[0024] The artificial neural network autoencoder (Autoencoders) model includes an input layer, a hidden layer, and an output layer connected in sequence;

[0025] The artificial neural network autoencoder model reconstructs input data through encoding and decoding operations, and aims to reduce reconstruction errors to achieve the purpose of feature extraction;

[0026] The artificial neural network autoencoder (Autoencoders) model is an unsupervised learning model. No data labels are used in the training process. The input data is used as the reconstruction target. The parameters inside the artificial neural network autoencoder (Autoencoders) model are gradually adjusted by minimizing the reconstruction error function. Several autoencoders are stacked to form stacked autoencoders.

[0027] The training process of the artificial neural network autoencoder (Autoencoders) model first trains each autoencoder one by one, and then fine-tunes the network through the back-propagation algorithm. Different autoencoder networks must ensure that the dimensions after mapping are the same.

[0028] Furthermore, in step S400, the multimodal data is multi-source heterogeneous data, and the multi-source heterogeneous data at least includes numerical data, image data, and classification data, wherein:

[0029] For numerical multi-source heterogeneous data, the numerical multi-source heterogeneous data is standardized so that the mean of the numerical multi-source heterogeneous data is 0 and the standard deviation is 1, so as to meet the input requirements of the stacked autoencoder. The corresponding linear transformation is performed after calculating the mean and standard deviation of the numerical multi-source heterogeneous data;

[0030] For multi-source heterogeneous image data, the image is grayed out and divided into image blocks of fixed size. Each image block is stretched into a one-dimensional vector as one of the input features of the stacked autoencoder. Multiple image blocks form a high-dimensional input matrix of the image data.

[0031] For multi-source heterogeneous data of categorical data type, independent coding method is used for conversion;

[0032] The processed multi-source heterogeneous data is input into a stacked autoencoder, which is composed of multiple autoencoders stacked in sequence. The first autoencoder performs preliminary feature extraction and compression on the input data.

[0033] For example, the combined input data composed of the original high-dimensional numerical and image multi-source heterogeneous data and the encoded categorical data multi-source heterogeneous data is compressed from a higher dimension (assumed to be n dimensions) to a lower dimension (m dimensions, m < n) to obtain an intermediate representation. The second autoencoder takes the output of the first autoencoder as input, further extracts more abstract features and compresses the dimension again, and so on.

[0034] During the training process, the parameters of the stacked autoencoder are adjusted by minimizing the reconstruction error, that is, making the output as close as possible to the original input data. After multiple iterative trainings, the stacked autoencoder learns the intrinsic feature representation in the multi-source heterogeneous data.

[0035] Finally, the trained stacked autoencoder is used to extract features from new multi-source heterogeneous data to obtain a feature representation.

[0036] Furthermore, the step S400 further includes:

[0037] During the manufacturing process, if it is necessary to complete the quality control of a certain process, it is necessary to analyze the factors affecting the quality of this process, and at the same time, it is also necessary to analyze the influencing factors of the previous process and / or the first process. Facing the product quality problems of multiple processes, it is necessary to establish a reasonable model to describe the influence of error sources on quality indicators in the case of multiple processes.

[0038] If a linear model is considered, the multi-process manufacturing process can be defined as:

[0039]

[0040] where B k0 is the offset matrix, B ij,k is the influence matrix, E k is the error matrix, whose mean is 0 and the error is For process k, there are q k possible inputs that can affect this process, expressed as:

[0041]

[0042] In a multi-process manufacturing process based on a linear model, the nonlinear relationship between different variables is considered. The kernel function can be used to map the nonlinear data to a high-dimensional data space so that it can be separable in the high-dimensional space, thereby transforming the nonlinear problem into a linear problem. The original nonlinear problem can be solved by solving the transformed linear problem.

[0043] Further, in step S500,

[0044] In the monitoring of a single process, the manufacturing process is monitored online based on the artificial neural network autoencoder model, which at least includes statistical analysis of the error sources, measuring the central tendency and dispersion of process parameters through mean, median, standard deviation, etc., and detecting the drift of process parameters at different times for the same process;

[0045] In addition, in the monitoring of a single process, virtual measurement can also be monitored. That is, when there is a lack of effective detection means at certain process nodes, resulting in the inability to perform actual measurement, the collected relevant parameters, such as the sensor parameters or other parameters of the equipment, are used to build models with the measurement results of interest. The measurement values ​​can be predicted based on the equipment sensor parameters in the future, without the need to perform actual measurements on specific equipment, so as to achieve real-time product quality prediction of the product in progress, machine performance monitoring and production process improvement. On this basis, a control chart is established to analyze whether the above variables are under control.

[0046] In cross-process monitoring, considering the abnormal situations that occur in adjacent processes, the mean of the same measurement value in different processes may change. In addition to the change in the mean, the covariance matrix of the variable also changes. Cross-process monitoring includes monitoring the change of the mean and correlation of multivariate data. Its hypothesis testing problem can be defined as:

[0047] H0:y i ~N(μ0,∑0)

[0048] Among them, μ0 is the mean of the data, and ∑0 is the covariance matrix of the data.

[0049] In addition, in order to fully consider the relationship between variables, that is, the change of one variable may be jointly determined by other variables, it is necessary to model the relationship between variables and define:

[0050] Y=f(X1,X2,...,X p )+ε

[0051] Among them, X1,X2,...,X p There are p variables, ε is the distribution of N(0,σ 2 ) error term.

[0052] In this way, the parameters between variables can be monitored across processes. In this process, control charts can be established for the measured values ​​and parameter distribution in the model to achieve online monitoring of the quality across processes at different times.

[0053] Further, in step S500,

[0054] In the monitoring of multiple processes, the quality monitoring of multiple processes monitors the product quality in the entire manufacturing process, makes informed decisions about product quality in real time, and establishes an effective quality monitoring model in the multistage manufacturing process (MMS). The modeling and analysis of multistage manufacturing data adopts the research method of quality prediction to achieve flexible prediction of product quality. For the overall prediction of multi-process quality, it is necessary to consider the time series characteristics and select the recurrent neural network (RNN) model or the Transformer model. The recurrent neural network (RNN) model or the Transformer model can extract information by adaptively selecting a series of positions or variables and give the importance of corresponding variables according to different tasks. First, a large amount of historical data is divided into training sets and test sets. By adjusting the structure and training parameters of the neural network, adjusting the structure of the neural network at least includes adjusting the number of hidden layers and the number of nodes, and adjusting the training parameters at least includes adjusting the learning rate and the number of iterations, the model is trained using the training set data to achieve a higher accuracy on the test set.

[0055] For the online monitoring of multi-process manufacturing process, assuming that an abnormality occurs in the τth process, the parameter problem of monitoring and fault identification is expressed as a multiple hypothesis testing problem, Θ is the model parameter that needs to be observed,

[0056] H0:Θ=Θ0,i=1,...,T

[0057]

[0058] Among them, Θ0 is the model parameter in the controlled state. In the alternative hypothesis, the model parameter changes after the τth process.

[0059] In the case of multiple faults, the test statistics reflecting the probability of fault occurrence in each process are obtained by integrating the offset direction information and the multivariate regression method. The control chart is designed based on the likelihood ratio test statistics to detect and identify multiple faults.

[0060] Online monitoring of multiple processes also includes the fusion analysis of multiple local statistics to achieve overall monitoring of the multi-process manufacturing process and determine which abnormal pattern causes the abnormal signal.

[0061] Furthermore, the present invention also proposes a multi-step manufacturing process quality monitoring device, which is used to execute the multi-step manufacturing process quality monitoring method. The multi-step manufacturing process quality monitoring device comprises:

[0062] A data acquisition unit, which acquires multimodal data and quality data from a multi-process manufacturing process based on intelligent sensors and data collection technology;

[0063] A data processing unit, the data processing unit pre-processes the multimodal data and the quality data and establishes quality checkpoints, the data processing unit is connected to the data acquisition unit;

[0064] A model training unit, wherein the model training unit establishes and trains an artificial neural network autoencoder (Autoencoders) model, and the model training unit is connected to the data processing unit;

[0065] A detection unit performs online monitoring on a single process, cross-processes and multiple processes and outputs diagnosis results and root cause analysis, and the detection unit is connected to the data processing unit.

[0066] Furthermore, the present invention also provides a computer-readable storage medium having program instructions stored thereon, and the program instructions, when executed by a processor, implement the multi-step manufacturing process quality monitoring method.

[0067] The beneficial effects of the present invention are:

[0068] The multi-process manufacturing process quality monitoring method, device and storage medium are based on the characteristics of the multi-process manufacturing process, establish a comprehensive online monitoring system, guide the quality process control of the production process in real time online, diagnose abnormal situations, improve problem handling and decision-making capabilities, and consider the characteristics of actual data in actual industrial application scenarios to develop solutions that are more suitable for the research problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 The figure is an overall flow chart of the quality monitoring method for multi-process manufacturing.

[0070] Figure 2 The present invention is a flow chart of a multi-step manufacturing process in a multi-step manufacturing process quality monitoring method.

[0071] Figure 3 It is a multi-process quality monitoring flow chart in the multi-process manufacturing process quality monitoring method.

[0072] Figure 4 Schematic diagram of the multi-process manufacturing path in the multi-process manufacturing process quality monitoring method.

[0073] Figure 5 It is a schematic diagram of a multi-process manufacturing process quality monitoring method in a multi-process manufacturing process quality monitoring method.

[0074] Figure 6 It is a schematic diagram of the SPC-based quality monitoring method in the multi-step manufacturing process quality monitoring method. DETAILED DESCRIPTION

[0075] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0076] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, top, bottom, etc. used in the present invention are only relative to the relative positional relationship of the components of the present invention in the drawings.

[0077] In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art. The terms used in this specification are only for describing specific embodiments, not for limiting the present invention. The term "and / or" used herein includes any combination of one or more related listed items.

[0078] It should be understood that, although the terms first, second, third, etc. may be used to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element.

[0079] Reference Figures 1 to 6In some embodiments, the technical solution of the present invention is a multi-process manufacturing process quality monitoring method, which is applied to a multi-process manufacturing process quality monitoring device, which includes a data acquisition unit, which is based on intelligent sensors and data collection technology to obtain multimodal data and quality data from the multi-process manufacturing process; a data processing unit, which pre-processes the multimodal data and quality data and establishes quality checkpoints, and the data processing unit is connected to the data acquisition unit; a model training unit, which establishes and trains an artificial neural network autoencoder (Autoencoders) model, and the model training unit is connected to the data processing unit; a detection unit, which performs online monitoring of a single process, cross-process and multiple processes and outputs diagnosis results and root cause analysis, and the detection unit is connected to the data processing unit, refer to Figure 1 The multi-step manufacturing process quality monitoring method comprises the following steps:

[0080] S100, based on intelligent sensors and data collection technology, obtains multimodal data and quality data from multi-process manufacturing processes;

[0081] S200, preprocessing the multimodal data to process outliers and missing values ​​in the multimodal data;

[0082] S300, determine the manufacturing process that has the greatest impact on the quality of the final product and establish quality checkpoints in key processes;

[0083] S400, establish and train artificial neural network autoencoder (Autoencoders) model to fuse and extract features of multimodal data;

[0084] S500, based on the artificial neural network autoencoder (Autoencoders) model, establish single-process, cross-process and multi-process online monitoring systems respectively;

[0085] S600: diagnose and analyze the root causes of abnormal problems found during monitoring, and update the detection strategy and artificial neural network autoencoder (Autoencoders) model as needed.

[0086] The beneficial effects of the present invention are:

[0087] The multi-process manufacturing process quality monitoring method, device and storage medium are based on the characteristics of the multi-process manufacturing process, establish a comprehensive online monitoring system, guide the quality process control of the production process in real time online, diagnose abnormal situations, improve problem handling and decision-making capabilities, and consider the characteristics of actual data in actual industrial application scenarios to develop solutions that are more suitable for the research problems.

[0088] Specifically, the present invention first fuses and extracts features from multimodal data, and selects key processes and key factors. On this basis, an online monitoring system of "single process-cross-process-multi-process" is established, a practical monitoring plan is proposed, and abnormal problems found in the monitoring are diagnosed and root cause analyzed, and finally the detection strategy and model are updated. The present invention models the multi-process manufacturing process through statistical, data mining and machine learning methods, improves the monitoring system of the manufacturing process, solves problems in the actual production of enterprises, and opens up new methods for quality control of multi-process manufacturing processes. Different from traditional research focusing on the quality monitoring problem of single-process manufacturing, this study establishes a comprehensive quality monitoring system for multi-process manufacturing processes, comprehensively diagnoses abnormal problems, and finds the root causes of the problems. Based on the above-mentioned statistical modeling and online monitoring methods based on multi-process manufacturing processes, solutions are proposed for the data characteristics in actual application scenarios.

[0089] Further, in step S100, the multimodal data includes at least maintenance records, state variables, process parameters, intermediate product characteristics, final product quality, and environmental sensor variables;

[0090] Determine quality data based on expert knowledge and previous manufacturing experiments, use different collection methods according to the data interface of different collection objects, and record quality data in real time;

[0091] In step S200, for abnormal values ​​of multimodal data, by setting a threshold and interval, abnormal data points beyond the interval are judged and eliminated; for local missing data caused by a temporary failure of the detection equipment, interpolation processing is performed; for data with different dimensions, data with a unified scale are standardized;

[0092] For missing values ​​of multimodal data, the missing data are processed by interpolation. The interpolation methods include at least single interpolation methods such as deletion, mean, random regression interpolation and regression interpolation, as well as multiple interpolation methods such as Markov chain Monte Carlo (MCMC) method and expectation maximization (EM) algorithm to interpolate missing values.

[0093] Corresponding to step S100, in semiconductor manufacturing, data can be collected in a variety of ways. In the photolithography process, the built-in sensor of the photolithography equipment is used to collect data such as exposure dose and focal length; in the etching process, parameters such as etching rate and etching selectivity are collected. At the same time, for physical properties such as the flatness and impurity concentration of the wafer surface, atomic force microscopes, secondary ion mass spectrometers and other detection equipment can be used to obtain data. These data will be continuously collected and recorded during the production of each wafer and the processing of each batch.

[0094] Corresponding to step S200, some abnormal values ​​may appear in the data. For example, in the data collected by the sensor, noise may be generated due to factors such as equipment failure and environmental interference. Sometimes the data may contain logical errors, so these abnormal data need to be preprocessed. Taking the wafer doping concentration data as an example, the measuring equipment may generate abnormal values ​​due to factors such as environmental interference. If the normal doping concentration range is within a certain interval, the obvious abnormal data points beyond the interval are judged and eliminated by setting a threshold. For local missing data caused by a short-term failure of the detection equipment, interpolation processing can be performed based on the doping concentration data of other positions of the same wafer or similar wafers in the same batch. In addition, the data dimensions obtained by different detection equipment may be different. For example, the wafer thickness is in nanometers, and the resistivity has its own unit. These data need to be standardized so that they can have a unified scale in subsequent analysis.

[0095] Corresponding to step S200, missing data is a common problem in quality monitoring, such as limited capacity of measuring equipment, long time consumption, resulting in low measurement and detection rates, and errors in machines or measuring equipment. Missing data may lead to bias in estimating model parameters and information loss. Interpolation of missing data includes single interpolation methods such as deletion, mean, random regression interpolation and regression interpolation, as well as multiple interpolation techniques such as Markov chain Monte Carlo (MCMC) method and expectation maximization (EM) algorithm.

[0096] Further, the step S300 includes:

[0097] S310, determining possible factors as target variables;

[0098] S320, screen key paths, variables, equipment, processes and parameters through evaluation and adaptability testing;

[0099] S330. Sort the candidate causes of the abnormality, screen the faults in the manufacturing system one by one, and take measures on the equipment as soon as possible.

[0100] Reference Figure 4In a multi-process manufacturing process, not all manufacturing processes have the same impact on the quality of the final product. Therefore, it is necessary to identify the manufacturing processes that have the greatest impact on the quality of the final product, and then establish quality checkpoints in these key processes to predict the possibility of producing non-defective products. It is neither necessary to check and predict quality only in the final manufacturing process, nor to give all processes the same attention to achieve the goal of minimizing costs while ensuring quality. First, it is necessary to identify possible factors as target variables, and then screen out key paths, variables, equipment, processes, and parameters through repeated evaluations and adaptability tests conducted by domain experts. And sort the candidate causes of the anomaly, screen the faults in the manufacturing system one by one, and take measures on the equipment as soon as possible.

[0101] Reference Figure 4 , showing a path for multi-process manufacturing. Due to the redundant characteristics of process variables, it is necessary to calculate the importance of critical paths, critical processes, and key variables, and sort them, and then set a significance level to determine the significant processes. For example, different products in multi-process manufacturing correspond to different manufacturing paths, and the critical paths, key processes, and key equipment may have a significant impact on product quality. This step helps to increase production and reduce resource and time waste.

[0102] Further, in step S400,

[0103] The artificial neural network autoencoder (Autoencoders) model includes an input layer, a hidden layer, and an output layer connected in sequence;

[0104] The artificial neural network autoencoder model reconstructs input data through encoding and decoding operations, and aims to reduce reconstruction errors to achieve the purpose of feature extraction;

[0105] The artificial neural network autoencoder (Autoencoders) model is an unsupervised learning model. No data labels are used in the training process. The input data is used as the reconstruction target. The parameters inside the artificial neural network autoencoder (Autoencoders) model are gradually adjusted by minimizing the reconstruction error function. Several autoencoders are stacked to form stacked autoencoders.

[0106] The training process of the artificial neural network autoencoder (Autoencoders) model first trains each autoencoder one by one, and then fine-tunes the network through the back-propagation algorithm. Different autoencoder networks must ensure that the dimensions after mapping are the same.

[0107] Specifically, in step S400, a data modeling operation is performed. A large amount of multimodal data is collected during the multi-process manufacturing process, such as time domain data such as temperature, rotation speed, pressure, and image data such as surface defects and roughness. The structural dimensions of these data are different, and it is necessary to effectively fuse and extract features from complex structural data. Since the feature mapping of artificial neural networks has strong inclusiveness for data structure forms and flexible network structure composition, multimodal information can be fused using an artificial neural network autoencoder (Autoencoders) model. The autoencoder consists of an input layer, a hidden layer, and an output layer, and is one of the classic models of deep learning. It reconstructs input data through encoding and decoding operations, and achieves the purpose of feature extraction by reducing the reconstruction error as the goal. Since data labels are not used in the training process, but only the input data is used as the reconstruction target, it is a typical unsupervised learning. By minimizing the reconstruction error function to gradually adjust the parameters inside the network, a number of autoencoders are stacked up to form stacked autoencoders. The training process is first to train the autoencoders one by one, and then fine-tune the entire network through the back propagation algorithm. Different autoencoder network structures can be different, as long as the dimensions after mapping are the same.

[0108] In the semiconductor manufacturing process, a large amount of multi-source heterogeneous data from different equipment and detection links is generated. For example, the lithography equipment of semiconductor chips generates lithography process parameter data, including numerical data such as exposure time, photoresist thickness, and light source intensity; at the same time, the electron microscope's detection of the chip's microstructure will generate image data, such as microscopic images of chip circuits; in addition, there is also classified data from the chip performance test link, such as whether the chip passes a specific functional test (qualified / unqualified). These different types of data have different structures, representations, and semantic information, but are closely related to the manufacturing quality and performance of semiconductors. Traditional data analysis methods have difficulty in effectively integrating and utilizing these multi-source heterogeneous data, and cannot fully tap the potential value in the data to improve the efficiency of semiconductor manufacturing and product quality.

[0109] Furthermore, in step S400, the multimodal data is multi-source heterogeneous data, and the multi-source heterogeneous data at least includes numerical data, image data, and classification data, wherein:

[0110] For numerical multi-source heterogeneous data, the numerical multi-source heterogeneous data is standardized so that the mean of the numerical multi-source heterogeneous data is 0 and the standard deviation is 1, so as to meet the input requirements of the stacked autoencoder. The corresponding linear transformation is performed after calculating the mean and standard deviation of the numerical multi-source heterogeneous data;

[0111] For image - type multi - source heterogeneous data, the images are grayscaled and segmented into image blocks of a fixed size. Each image block is stretched into a one - dimensional vector, which serves as one of the input features of the stacked auto - encoder. Multiple image blocks form the high - dimensional input matrix of the image data;

[0112] For classification - data - type multi - source heterogeneous data, an independent encoding method is used for conversion;

[0113] The processed multi - source heterogeneous data is input into the stacked auto - encoder. The stacked auto - encoder is composed of multiple auto - encoders stacked in sequence. The first auto - encoder performs preliminary feature extraction and compression on the input data,

[0114] For example, the combined input data composed of original high - dimensional numerical and image - type multi - source heterogeneous data and encoded classification - data - type multi - source heterogeneous data is compressed from a higher dimension (assumed to be n - dimension) to a lower dimension (m - dimension, m < n) to obtain an intermediate representation. The second auto - encoder takes the output of the first auto - encoder as input, further extracts more abstract features and compresses the dimension again, and so on;

[0115] During the training process, the parameters of the stacked auto - encoder are adjusted by minimizing the reconstruction error, that is, making the output as close as possible to the original input data. After multiple iterative trainings, the stacked auto - encoder learns the intrinsic feature representation in the multi - source heterogeneous data;

[0116] Finally, the trained stacked auto - encoder is used to extract features from new multi - source heterogeneous data to obtain the feature representation.

[0117] In a specific embodiment, for numerical lithography process parameter data, it is standardized so that its mean is 0 and its standard deviation is 1 to meet the input requirements of the stacked auto - encoder. For example, for a set of exposure time data [10.5, 11.2, 9.8, 10.1, 12.0], corresponding linear transformation is performed after calculating its mean and standard deviation.

[0118] In a specific embodiment, for image - type chip microstructure data, the images are grayscaled and segmented into image blocks of a fixed size. For example, after grayscaling a 1024×1024 - pixel chip micro - image, it is segmented into 32×32 - pixel image blocks. Then, each image block is stretched into a one - dimensional vector, which serves as one of the input features of the stacked auto - encoder. Assuming that each image block is stretched into a 1024 - dimensional vector after stretching, multiple image blocks form the high - dimensional input matrix of the image data.

[0119] In a specific embodiment, for categorical data (such as chip test results), one-hot encoding is used for conversion. For example, if there are two categories of qualified and unqualified, the qualified encoding is [1, 0], and the unqualified encoding is [0, 1].

[0120] The processed multi-source heterogeneous data is input into a stacked autoencoder. The stacked autoencoder is composed of multiple autoencoders stacked in sequence. The first autoencoder performs preliminary feature extraction and compression on the input data. For example, the combined input data consisting of original high-dimensional numerical and image data and the encoded categorical data is compressed from a higher dimension (assumed to be n dimensions) to a lower dimension (m dimensions, where m < n) to obtain an intermediate representation. The second autoencoder takes the output of the first autoencoder as input, further extracts more abstract features and compresses the dimension again, and so on. During the training process, the parameters of the stacked autoencoder are adjusted by minimizing the reconstruction error. That is, the output is made as close as possible to the original input data. After multiple iterative trainings, the stacked autoencoder learns the intrinsic feature representations in the multi-source heterogeneous data. Finally, the trained stacked autoencoder is used to process the multi-source heterogeneous data in the new semiconductor manufacturing process, and the obtained feature representations can be used for various applications, such as fault prediction, product quality assessment, and production process optimization in the semiconductor manufacturing process.

[0121] Furthermore, step S400 further includes:

[0122] During the manufacturing process, if it is necessary to complete the quality control of a certain process, it is necessary to analyze the factors affecting the quality of this process, and at the same time, it is also necessary to analyze the influencing factors of the previous process and / or the first process. Facing the product quality problems of multiple processes, a reasonable model needs to be established to describe the influence of error sources on quality indicators in the case of multiple processes;

[0123] If a linear model is considered, the multi-process manufacturing process can be defined as:

[0124]

[0125] where B k0 is the offset matrix, B ij,k is the influence matrix, E k is the error matrix, whose mean is 0 and the error is For process k, there are q k possible inputs that may affect this process, which are expressed as:

[0126]

[0127] In a multi-process manufacturing process based on a linear model, the nonlinear relationship between different variables is considered. The kernel function can be used to map the nonlinear data to a high-dimensional data space so that it can be separable in the high-dimensional space, thereby transforming the nonlinear problem into a linear problem. The original nonlinear problem can be solved by solving the transformed linear problem.

[0128] In a specific embodiment, in process k, the quality data characteristics of the previous k-1 processes are considered to represent a specific quality parameter. In semiconductor data, the final quality of the chip may be affected by the quality data of the circuit probe test and the final test. By fitting a large amount of experimental data, a linear regression model can be established to obtain the matrix value in the above formula. After using the kernel function to build a nonlinear model, we still need to train the model to determine the coefficients. Usually, methods such as least squares or gradient descent are used to minimize the loss function (such as mean square error). To evaluate the performance of the model, we can use some evaluation indicators, such as mean square error (MSE), determination coefficient, etc. If the mean square error is small, it also means that the prediction accuracy of the model is high; if the determination coefficient is close to 1, it means that the model fits well.

[0129] In addition, based on this linear model, we can consider the nonlinear relationship between different variables. For example, we can use a kernel function to map nonlinear data to a high-dimensional data space, making it separable in the high-dimensional space, thereby transforming the nonlinear problem into a linear problem, and solving the original nonlinear problem by solving the transformed linear problem. For example, the Gaussian kernel function is a commonly used kernel function that assigns weights based on the distance between data points. The result of this kernel function is used as a new independent variable to construct a linear regression model. Then, we use the linear regression model to fit the data. In this model, linear regression is actually performed in a high-dimensional space, while it looks like a nonlinear model in the original space.

[0130] Furthermore, in the step S500, in the monitoring of a single process, the manufacturing process is monitored online based on an artificial neural network autoencoder (Autoencoders) model, which at least includes statistical analysis of error sources, measuring the central tendency and dispersion of process parameters by means of mean, median, standard deviation, etc., and detecting the drift of process parameters at different times for the same process;

[0131] In addition, in the monitoring of a single process, virtual measurement can also be monitored. That is, when there is a lack of effective detection means at certain process nodes, which makes it impossible to perform actual measurement, the collected relevant parameters, such as the sensor parameters or other parameters of the equipment, are used to build models with the measurement results of interest. The measurement values ​​can be predicted based on the equipment sensor parameters in the future, without the need to perform actual measurements on specific equipment, so as to achieve real-time product quality prediction of the product in progress, machine performance monitoring and production process improvement. On this basis, a control chart is established to analyze whether the above variables are under control.

[0132] In cross-process monitoring, considering the abnormal situations that occur in adjacent processes, the mean of the same measurement value in different processes may change. In addition to the change in the mean, the covariance matrix of the variable also changes. Cross-process monitoring includes monitoring the change of the mean and correlation of multivariate data. Its hypothesis testing problem can be defined as:

[0133] H0:y i ~N(μ0,∑0)

[0134] Among them, μ0 is the mean of the data, and ∑0 is the covariance matrix of the data.

[0135] In addition, in order to fully consider the relationship between variables, that is, the change of one variable may be jointly determined by other variables, it is necessary to model the relationship between variables and define:

[0136] Y=f(X1,X2,...,X p )+ε

[0137] Among them, X1,X2,...,X p There are p variables, and ε is distributed as N(0,σ 2 ) error term.

[0138] In this way, the parameters between variables can be monitored across processes. In this process, control charts can be established for the measured values ​​and parameter distribution in the model to achieve online monitoring of the quality across processes at different times.

[0139] Further, in step S500,

[0140] In the monitoring of multiple processes, the quality monitoring of multiple processes monitors the product quality in the entire manufacturing process, makes informed decisions about product quality in real time, and establishes an effective quality monitoring model in the multistage manufacturing process (MMS). The modeling and analysis of multistage manufacturing data adopts the research method of quality prediction to achieve flexible prediction of product quality; for the overall prediction of multiple processes of quality, it is necessary to consider the time series characteristics, and select the recurrent neural network (RNN) model or the Transformer model. The recurrent neural network (RNN) model or the Transformer model can extract information by adaptively selecting a series of positions or variables, and give the importance of corresponding variables according to different tasks;

[0141] Firstly, a large amount of historical data is divided into training set and test set. By adjusting the structure and training parameters of the neural network, the adjustment of the structure of the neural network at least includes adjusting the number of hidden layers and the number of nodes, and the adjustment of the training parameters at least includes adjusting the learning rate and the number of iterations, the model is trained using the training set data to achieve a high accuracy rate on the test set. For the online monitoring of multi-process manufacturing processes, it is assumed that an abnormality occurs in the τth process, and the parameter problem of monitoring and fault identification is expressed as a multiple hypothesis testing problem. Θ is the model parameter that needs to be observed.

[0142] H0:Θ=Θ0,i=1,...,T

[0143]

[0144] Among them, Θ0 is the model parameter in the controlled state. In the alternative hypothesis, the model parameter changes after the τth process.

[0145] In the case of multiple occurrences of faults, by integrating the offset direction information and the multivariate regression method, test statistics reflecting the probability of fault occurrence in each process are obtained, and control charts are designed based on the likelihood ratio test statistics to detect and identify multiple occurrences of faults;

[0146] Online monitoring of multiple processes also includes the fusion analysis of multiple local statistics to achieve overall monitoring of the multi-process manufacturing process and determine which abnormal pattern causes the abnormal signal.

[0147] Specifically, in step S500, based on the artificial neural network autoencoder (Autoencoders) model, refer to Figure 5 , and proposed an online monitoring method of "single process-cross process-multiple processes". Figure 6, which is a specific quality monitoring method based on control charts.

[0148] In practical applications, monitoring models should be updated regularly to handle any new patterns and any changes in data characteristics, and the monitoring strategy should be optimized and adjusted to reduce the chances of performance degradation and misclassification.

[0149] Reference Figure 5 The quality monitoring method of "single process-cross-process-multiple processes" is that in the monitoring of a single process, the manufacturing process can be monitored online based on the results of the above data modeling. For example, the error source can be statistically analyzed (such as mean, median, standard deviation, etc., to understand the concentration trend and dispersion of process parameters), and the drift of process parameters at different times for the same process can be detected. In addition, virtual measurement can also be monitored, that is, when there is a lack of effective detection means at some process nodes, resulting in the inability to perform actual measurement, the collected relevant parameters, such as the sensor parameters or other parameters of the equipment, are used to model the measurement results of interest. This model can then predict the measurement value based on the equipment sensor parameters without the need for actual measurement on a specific device, so as to achieve real-time product quality prediction, machine performance monitoring and production process improvement of the product in progress. On this basis, a control chart is established to analyze whether the above variables are under control. Taking the photolithography process as an example, key quality indicators such as the critical dimension (CD) of the photolithography pattern are selected as the monitoring object. First, the mean and standard deviation of CD are calculated based on historical data. Then, draw an XR control chart, where the X chart is used to monitor the mean change of CD, and the R chart is used to monitor the range (difference between the maximum and minimum values) of CD. During the production process, the CD of the photolithography pattern of a batch of wafers is measured at a certain interval (such as every hour), and the measured values ​​are plotted on the control chart. If multiple consecutive points (such as 7 points) are found to appear on one side of the center line of the X chart, or one point exceeds the upper or lower control limit, this indicates that there may be an abnormality in the photolithography process. For example, when it is found that the CD value measured at a certain moment exceeds the upper limit of the X chart, the system will immediately trigger an alarm, indicating that there may be problems such as excessive exposure dose or abnormal photoresist performance. At this time, the process parameters can be adjusted in time or the equipment can be inspected and maintained to ensure the quality stability of the photolithography process.

[0150] Online monitoring across processes takes into account abnormal situations that occur in adjacent processes. The mean of the same measurement value may change in different processes. In addition to the change in the mean, the covariance matrix of the variable also changes. The monitoring here includes the change monitoring of the mean and correlation of multivariate data. The hypothesis test problem can be defined as:

[0151] H0:y i ~N(μ0,∑0)

[0152] Where μ0 is the mean of the data, and ∑0 is the covariance matrix of the data. Collect a certain amount of sample data from a new production batch (such as the lithography and etching critical dimension data of the first 10 wafers) and calculate the sample mean and sample variance. Use an appropriate test statistic and determine the rejection region based on the set significance level (such as 0.05). If the calculated value falls into the rejection region, the null hypothesis is rejected, indicating that the change in the mean of the measured value is abnormal.

[0153] As production proceeds, the corresponding statistics can be plotted on the control chart according to the selected control chart. If multiple consecutive points (such as 7 points) appear on one side of the center line, or one point exceeds the control limit, it means that the measurement values ​​​​across processes are abnormal.

[0154] In addition, in order to fully consider the relationship between variables, that is, the change of one variable may be determined by other variables, the relationship between variables needs to be modeled. For example, define:

[0155] Y=f(X1,X2,...,X p )+ε

[0156] Among them, X1,X2,...,X p There are p variables, and ε is distributed as N(0,σ 2 ). Thus, the parameters between variables can be monitored across processes. In this process, control charts can be established for the distribution of measured values ​​and parameters in the model to achieve online monitoring of the quality across processes at different times.

[0157] The purpose of multi-process quality monitoring is to monitor product quality throughout the manufacturing process, make informed decisions about product quality in real time, and establish an effective quality monitoring model in the MMS. The SPC method used in traditional quality control does not have the ability to predict quality and cannot prevent quality problems in advance. Based on this, the modeling and analysis of multi-process manufacturing data adopts the research method of quality prediction to achieve flexible prediction of product quality. For the overall prediction of multi-process quality, it is necessary to consider the time series characteristics. A recurrent neural network (RNN) model or a Transformer model can be used, which can extract information by adaptively selecting a series of positions or variables and give the importance of corresponding variables according to different tasks. First, a large amount of historical data is divided into training sets and test sets, such as 80% of the data for training and 20% of the data for testing. By adjusting the structure of the neural network (such as the number of hidden layers and nodes) and training parameters (such as learning rate and number of iterations), the model is trained using the training set data to achieve a high accuracy rate on the test set. In the actual production process, when a new product enters the production line, the relevant data is input into the trained neural network model to achieve quality prediction.

[0158] For online monitoring of multi-process manufacturing processes, we can assume that an anomaly occurs in the τth process, so the parameter problem of monitoring and fault identification can be formulated as a multiple hypothesis testing problem, where Θ is the model parameter that needs to be observed.

[0159] H0:Θ=Θ0,i=1,...,T

[0160]

[0161] Where Θ0 is the model parameter in the controlled state. In the alternative hypothesis, the model parameter changes after the τth process. In the case of multiple failures, by integrating the offset direction information and the multivariate regression method, test statistics reflecting the probability of failure in each process can be obtained. Based on the likelihood ratio test statistic, a control chart can be designed to detect and identify multiple failures. (The test hypothesis and control chart method are similar to the above and are described below. Figure 6 It is explained in the following: The verification method of the test hypothesis involves using an appropriate test statistic and determining the rejection region based on the set significance level. If the calculated test statistic falls into the rejection region, the null hypothesis is rejected and the multi-process manufacturing process is abnormal. The control chart method involves selecting an appropriate control chart, calculating the test statistic, plotting the statistics at different times on the control chart, and determining the upper and lower control lines on the center line. If the statistic exceeds the control limit, an abnormality is indicated.)

[0162] Online monitoring of multiple processes also involves the fusion analysis of multiple local statistics to achieve overall monitoring of the multi-process manufacturing process. This can be set up through professional background knowledge or by setting the preference for different abnormal modes. This fusion method is very helpful for abnormal diagnosis and can determine which abnormal mode causes the abnormal signal.

[0163] Reference Figure 6 , which is a quality monitoring flow chart based on statistical process control (SPC). First, it is necessary to define the quality indicators that need to be monitored, and select reasonable hypothesis testing problems and corresponding control chart methods. On this basis, the corresponding statistical parameters that need to be observed in the control chart are calculated using historical data and the data to be monitored, and the control chart is drawn. Set the center line, upper control line and lower control line to determine whether the control chart is in a controlled state. If it is in an uncontrolled state, it is necessary to diagnose the abnormality, including abnormal process, abnormal parameters, abnormal type and possible cause analysis, update the production process and process parameters, and achieve the purpose of improving product quality. Continue online monitoring after adjustment to ensure that the process quality returns to normal.

[0164] Furthermore, the present invention also proposes a multi-step manufacturing process quality monitoring device, which is used to execute the multi-step manufacturing process quality monitoring method. The multi-step manufacturing process quality monitoring device comprises:

[0165] A data acquisition unit, which acquires multimodal data and quality data from a multi-process manufacturing process based on intelligent sensors and data collection technology;

[0166] A data processing unit, the data processing unit pre-processes the multimodal data and the quality data and establishes quality checkpoints, the data processing unit is connected to the data acquisition unit;

[0167] A model training unit, wherein the model training unit establishes and trains an artificial neural network autoencoder (Autoencoders) model, and the model training unit is connected to the data processing unit;

[0168] A detection unit performs online monitoring on a single process, cross-processes and multiple processes and outputs diagnosis results and root cause analysis, and the detection unit is connected to the data processing unit.

[0169] Furthermore, the present invention also provides a computer-readable storage medium having program instructions stored thereon, and the program instructions, when executed by a processor, implement the multi-step manufacturing process quality monitoring method.

[0170] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of protection of the present disclosure. All should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A method for monitoring quality of a multi-process manufacturing process, wherein the method is applied to a multi-process manufacturing process quality monitoring device, wherein the multi-process manufacturing process quality monitoring device comprises a data acquisition unit, wherein the data acquisition unit acquires multimodal data and quality data from the multi-process manufacturing process based on intelligent sensors and data collection technology; A data processing unit, the data processing unit pre-processes the multimodal data and the quality data and establishes quality checkpoints, the data processing unit is connected to the data acquisition unit; A model training unit, wherein the model training unit establishes and trains an artificial neural network autoencoder model, and the model training unit is connected to the data processing unit; a detection unit, wherein the detection unit performs online monitoring of a single process, cross-process, and multiple processes and outputs a diagnosis result and a root cause analysis, and the detection unit is connected to the data processing unit, wherein the multi-process manufacturing process quality monitoring method comprises the following steps: S100, based on intelligent sensors and data collection technology, obtains multimodal data and quality data from multi-process manufacturing processes; S200, preprocessing the multimodal data to process outliers and missing values ​​in the multimodal data; S300, determine the manufacturing process that has the greatest impact on the quality of the final product and establish quality checkpoints in key processes; S400, establish and train an artificial neural network autoencoder model to fuse and extract features from multimodal data; S500, based on the artificial neural network autoencoder model, establish online monitoring systems for single process, cross-process and multi-process respectively; S600, diagnose and analyze the root causes of abnormal problems found during monitoring, and update the detection strategy and artificial neural network autoencoder model as needed.

2. The multi-step manufacturing process quality monitoring method according to claim 1 is characterized in that: In step S100, the multimodal data includes at least maintenance records, state variables, process parameters, intermediate product characteristics, final product quality, and environmental sensor variables; Determine quality data based on expert knowledge and previous manufacturing experiments, use different collection methods according to the data interface of different collection objects, and record quality data in real time; In step S200, for abnormal values ​​of the multimodal data, by setting a threshold and an interval, abnormal data points that exceed the interval are judged and removed; For local missing data caused by temporary failure of detection equipment, interpolation processing is performed; for data with different dimensions, data with unified scale are standardized; For missing values ​​of multimodal data, the missing data are processed by interpolation. The interpolation methods include at least single interpolation methods such as deletion, mean, random regression interpolation and regression interpolation, as well as Markov chain Monte Carlo method and expectation maximization algorithm multiple interpolation methods to interpolate missing values.

3. The multi-step manufacturing process quality monitoring method according to claim 1, characterized in that: The step S300 includes: S310, determining possible factors as target variables; S320, screen key paths, variables, equipment, processes and parameters through evaluation and adaptability testing; S330. Sort the candidate causes of the abnormality, screen the faults in the manufacturing system one by one, and take measures on the equipment as soon as possible.

4. The multi-step manufacturing process quality monitoring method according to claim 1, characterized in that: In the step S400, The artificial neural network autoencoder model includes an input layer, a hidden layer, and an output layer connected in sequence; The artificial neural network autoencoder model reconstructs input data through encoding and decoding operations, and aims to reduce reconstruction errors to achieve the purpose of feature extraction; The artificial neural network autoencoder model is an unsupervised learning model, which does not use data labels during the training process, takes input data as the reconstruction target, gradually adjusts the parameters inside the artificial neural network autoencoder model by minimizing the reconstruction error function, and stacks several autoencoders to form a stacked autoencoder; The training process of the artificial neural network autoencoder model is to first train each autoencoder one by one, and then fine-tune the network through the back-propagation algorithm. Different autoencoder networks need to ensure that the dimensions after mapping are the same.

5. The multi-step manufacturing process quality monitoring method according to claim 4 is characterized in that: In step S400, the multimodal data is multi-source heterogeneous data, and the multi-source heterogeneous data at least includes numerical data, image data, and classification data, wherein: For numerical multi-source heterogeneous data, the numerical multi-source heterogeneous data is standardized so that the mean of the numerical multi-source heterogeneous data is 0 and the standard deviation is 1, so as to meet the input requirements of the stacked autoencoder. The corresponding linear transformation is performed after calculating the mean and standard deviation of the numerical multi-source heterogeneous data; For multi-source heterogeneous image data, the image is grayed out and divided into image blocks of fixed size. Each image block is stretched into a one-dimensional vector as one of the input features of the stacked autoencoder. Multiple image blocks form a high-dimensional input matrix of the image data. For multi-source heterogeneous data of categorical data type, independent coding method is used for conversion; The processed multi-source heterogeneous data is input into the stacked autoencoder. The stacked autoencoder is composed of multiple autoencoders stacked in sequence. The first autoencoder performs preliminary feature extraction and compression on the input data. For example, the joint input data consisting of the original high-dimensional numerical and image-type multi-source heterogeneous data and the encoded categorical data-type multi-source heterogeneous data is compressed from a higher dimension to a lower dimension to obtain an intermediate representation. The second autoencoder uses the output of the first autoencoder as input to further extract more abstract features and compress the dimension again, and so on. During the training process, the parameters of the stacked autoencoder are adjusted by minimizing the reconstruction error, that is, making the output as close as possible to the original input data. After multiple iterations of training, the stacked autoencoder learns the intrinsic feature representation of multi-source heterogeneous data. Finally, the trained stacked autoencoder is used to extract features from the new multi-source heterogeneous data to obtain feature representation.

6. The multi-step manufacturing process quality monitoring method according to claim 1, characterized in that: The step S400 further includes: In the manufacturing process, if you want to complete the quality control of a certain process, you need to analyze the factors that affect the quality of this process, and you also need to analyze the factors that affect the previous process and / or the first process. In the face of product quality issues in multiple processes, you need to establish a reasonable model to describe the impact of error sources on quality indicators in the case of multiple processes; If a linear model is considered, the multi-step manufacturing process can be defined as: Among them, B k0 is the offset matrix, B ij,k is the influence matrix, E k is the error matrix, whose mean is 0 and the error is For process k, there is q k The inputs that may affect this process are expressed as: In a multi-process manufacturing process based on a linear model, the nonlinear relationship between different variables is considered. The kernel function can be used to map the nonlinear data to a high-dimensional data space so that it can be separable in the high-dimensional space, thereby transforming the nonlinear problem into a linear problem. The original nonlinear problem can be solved by solving the transformed linear problem.

7. The multi-step manufacturing process quality monitoring method according to claim 1, characterized in that: In step S500, In the monitoring of a single process, the manufacturing process is monitored online based on the artificial neural network autoencoder model, which at least includes statistical analysis of the error sources, measuring the central tendency and dispersion of process parameters through mean, median, standard deviation, etc., and detecting the drift of process parameters at different times for the same process; In addition, in the monitoring of a single process, virtual measurement can also be monitored. That is, when there is a lack of effective detection means at certain process nodes, which makes it impossible to perform actual measurement, the collected relevant parameters, such as the sensor parameters or other parameters of the equipment, are used to build models with the measurement results of interest. The measurement values ​​can be predicted based on the equipment sensor parameters in the future, without the need to perform actual measurements on specific equipment, so as to achieve real-time product quality prediction of the product in progress, machine performance monitoring and production process improvement. On this basis, a control chart is established to analyze whether the above variables are under control. In cross-process monitoring, considering the abnormal situations that occur in adjacent processes, the mean of the same measurement value in different processes may change. In addition to the change in the mean, the covariance matrix of the variable also changes. Cross-process monitoring includes monitoring the change of the mean and correlation of multivariate data. Its hypothesis testing problem can be defined as: H0: the i ~N(μ0,∑0) Among them, μ0 is the mean of the data, and ∑0 is the covariance matrix of the data. In addition, in order to fully consider the relationship between variables, that is, the change of one variable may be jointly determined by other variables, it is necessary to model the relationship between variables and define: Y=f(X1,X2,...,X p )+ε Among them, X1,X2,...,X p There are p variables, ε is the distribution of N(0,σ 2 ) error term. In this way, the parameters between variables can be monitored across processes. In this process, control charts can be established for the measured values ​​and parameter distribution in the model to achieve online monitoring of the quality across processes at different times.

8. The multi-step manufacturing process quality monitoring method according to claim 1, characterized in that: In step S500, In the monitoring of multiple processes, the quality monitoring of multiple processes monitors the product quality in the entire manufacturing process, makes informed decisions about product quality in real time, and establishes an effective quality monitoring model in the multi-process manufacturing system. The modeling and analysis of multi-process manufacturing data adopts the research method of quality prediction to achieve flexible prediction of product quality. For the overall prediction of multiple processes of quality, it is necessary to consider the time series characteristics and select a recurrent neural network model or a Transformer model. The recurrent neural network model or the Transformer model can extract information by adaptively selecting a series of positions or variables, and give the importance of corresponding variables according to different tasks. First, a large amount of historical data is divided into a training set and a test set. By adjusting the structure and training parameters of the neural network, the adjustment of the structure of the neural network at least includes adjusting the number of hidden layers and the number of nodes, and the adjustment of the training parameters at least includes adjusting the learning rate and the number of iterations, the model is trained using the training set data to achieve a higher accuracy rate on the test set; For the online monitoring of multi-process manufacturing process, assuming that an abnormality occurs in the τth process, the parameter problem of monitoring and fault identification is expressed as a multiple hypothesis testing problem, Θ is the model parameter that needs to be observed, H0:Θ=Θ0,i=1,...,T Among them, Θ0 is the model parameter in the controlled state. In the alternative hypothesis, the model parameter changes after the τth process. In the case of multiple faults, the test statistics reflecting the probability of fault occurrence in each process are obtained by integrating the offset direction information and the multivariate regression method. The control chart is designed based on the likelihood ratio test statistics to detect and identify multiple faults. Online monitoring of multiple processes also includes the fusion analysis of multiple local statistics to achieve overall monitoring of the multi-process manufacturing process and determine which abnormal pattern causes the abnormal signal.

9. A multi-step manufacturing process quality monitoring device, used to execute the multi-step manufacturing process quality monitoring method according to any one of claims 1 to 8, characterized in that: The multi-step manufacturing process quality monitoring device comprises: A data acquisition unit, which acquires multimodal data and quality data from a multi-process manufacturing process based on intelligent sensors and data collection technology; A data processing unit, the data processing unit pre-processes the multimodal data and the quality data and establishes quality checkpoints, the data processing unit being connected to the data acquisition unit; A model training unit, wherein the model training unit establishes and trains an artificial neural network autoencoder model, and the model training unit is connected to the data processing unit; A detection unit performs online monitoring on a single process, cross-processes and multiple processes and outputs diagnosis results and root cause analysis, and the detection unit is connected to the data processing unit. 10 . A computer-readable storage medium having program instructions stored thereon, wherein the program instructions are executed by a processor to implement the method according to claim 1 .

Images