A distributed parallel PCA process monitoring modeling method based on continuous mapreduce

Abstract

The invention discloses a distributed parallel PCA process monitoring modeling method based on continuous MapReduce, which belongs to the field of industrial process monitoring and control. This method gives a distributed parallel design scheme based on MapReduce, which includes using three MapReduces to realize the standardization of large-scale data, and based on the method of marking the elements required for matrix multiplication in advance, realizing large-scale matrix multiplication through one MapReduce and then calculating the correlation Variance matrix, and finally calculate T 2 When comparing SPE statistics, use MapReduce once to realize parallel multiplication of each sample and projection matrix. Through the above continuous MapReduce operation, some time-consuming operations that increase with the increase in sample size can be decomposed, and the operation can be completed in parallel, which can effectively improve PCA. Modeling speed for process monitoring models.

Description

technical field [0001] The invention belongs to the field of industrial process monitoring and control, and relates to a distributed parallel PCA process monitoring modeling method based on continuous MapReduce. Background technique [0002] In the industrial production process, due to the aging of the machine itself or the interference of external factors, production failures often occur, and the most intuitive manifestation of failures is the change of quality variables. The fluctuation of quality variables will directly affect the products produced, resulting in a significant decline in product quality and great losses to industrial enterprises. Therefore, it is very necessary to monitor industrial production process faults. [0003] The data-driven fault monitoring method is very commonly used at present. Since the quality variables will change correspondingly in the data after the fault occurs, if the fault monitoring model can be established based on the normal quality...

AI Technical Summary

Problems solved by technology

[0004] At present, with the continuous development of modern industry and computer technology, more and more data can be accumulated in the industrial process, and a series of industrial big data problems have gradually formed. The traditional data-driven process monitoring model is no matter in time or effect. It is difficult to apply on large-scale data, including the PCA model for process monitoring. From the above introduction, it can be seen that the PCA model mainly standardizes the data first, and then obtains the projection vector by decomposing the eigenvalues ​​of the data covariance matrix , and finally use the projection vector to multiply each sample to obtain the pivot. Therefore, in the above steps, the standardization of the data needs to calculate the mean and standard deviation first, and then process each sample successively. If there is a large amount of data It will be very time-consuming; when calculating the covariance matrix, the idea of ​​multiplying the standardized data matrix is ​​adopted, and it will also generate a lot of time overhead when calculating the multiplication of two very large-scale matrices; however, in the feature In the value decomposition stage, since the order of the covariance matrix is ​​the number of variables of a single sample, the number of variables is generally much smaller than the number of samples, so the time overhead of this stage will not be very large; then use the extracted eigenvectors to form The projection matrix calculates the T corresponding to each sample 2 Statistics, and when calculating the sample reconstruction value based on the pivot and projection matrix, since it is still necessary to multiply the projection matrix with the sample or the pivot of each sample one by one, it is calculated when the amount of data is large will be time consuming

Images