Industrial process fault detection method and system for missing data

Abstract

The invention provides an industrial process fault detection method and system for missing data, and relates to the technical field of industrial process fault detection, and the method comprises the steps: obtaining data containing a missing value under a normal working condition as training data; carrying out filling processing on missing values in the training data; determining an optimal solution of the training data after filling processing by minimizing the detection index; calculating statistics of variables according to the optimal solution of the training data after filling processing; determining a control limit according to the statistics of the variables of the training data and the confidence level; acquiring real-time data containing missing values; carrying out filling processing on missing values in the real-time data; determining an optimal solution of the complete data after filling processing by minimizing the detection index; calculating the statistics of the variables according to the optimal solution of the complete data after filling processing; calculating a detection index according to the statistics of the variables of the real-time data; and under the condition that the detection index is greater than the control limit, judging that the working condition is an abnormal working condition.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial process fault detection, and in particular to an industrial process fault detection method and system for missing data. Background Art

[0002] With the development of economy and the continuous expansion of industrial production, ensuring the safety of production process and controlling product quality have become important challenges. In the actual production process, many serious accidents often originate from some minor faults, and the early detection of these faults is crucial to ensure the normal operation of the production process and improve product quality. In the production process, a large number of sensors are responsible for measuring various process variables, and the readings of these sensors directly reflect the status of the production process. However, in long-term operation, the accuracy of the sensor will decrease, and drift, deviation or complete failure may occur, resulting in data loss and anomalies. In addition, data loss may also occur during data entry and storage. These factors often lead to varying degrees of missing in the acquired production data. Therefore, research on early fault detection for missing data is of great significance to improving the automation level of the production process, reducing manual intervention and ensuring production safety.

[0003] In current technology, methods for dealing with missing data mainly include data recovery methods of low-rank algorithms and other technologies. Low-rank algorithms assume that the matrix has a low-rank or near-low-rank structure and recover the data by finding a low-rank matrix consistent with the incomplete matrix. Other methods include using Bayesian statistical models and maximizing the likelihood function to infer missing data.

[0004] However, these methods suffer from high false positive rates in practical applications, cannot fully restore data integrity, and cannot effectively detect early failures. Summary of the invention

[0005] In order to solve the technical problems that traditional missing data processing methods have high false alarm rate in practical applications, cannot fully restore data integrity, and cannot effectively detect early faults, the present invention provides an industrial process fault detection method and system for missing data.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] An industrial process fault detection method for missing data provided by an embodiment of the present invention includes:

[0009] S1: Obtain data with missing values ​​under normal working conditions as training data;

[0010] S2: Filling missing values ​​in the training data;

[0011] S3: Determine the optimal solution for the padded training data by minimizing the detection index;

[0012] S4: Calculate the statistics of the variables according to the optimal solution of the training data after filling;

[0013] S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level;

[0014] S6: Obtain real-time data with missing values;

[0015] S7: Filling missing values ​​in the real-time data;

[0016] S8: Determine the optimal solution for the complete data after filling by minimizing the detection index;

[0017] S9: Calculate the statistics of the variables according to the optimal solution of the complete data after filling processing;

[0018] S10: Calculate the detection index according to the statistics of the variables of the real-time data;

[0019] S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

[0020] Second aspect:

[0021] An industrial process fault detection system for missing data provided by an embodiment of the present invention includes: a memory and one or more processors;

[0022] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the above-mentioned industrial process fault detection method for missing data.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] In the present invention, the optimal solution of the filled training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated according to the optimal solution of the filled training data, and the control limit is determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the filled complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a process flow of an industrial process fault detection method for missing data provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the structure of an industrial process fault detection system for missing data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0030] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0031] Reference Manual Attached Figure 1 , shows a flow chart of an industrial process fault detection method for missing data provided by an embodiment of the present invention.

[0032] An embodiment of the present invention provides an industrial process fault detection method for missing data, which can be implemented by an industrial process fault detection device for missing data, and the industrial process fault detection device for missing data can be a terminal or a server. The processing flow of the industrial process fault detection method for missing data may include the following steps:

[0033] S1: Obtain data with missing values ​​under normal working conditions as training data.

[0034] It should be noted that normal operating conditions refer to industrial data collected under normal operating conditions. In actual industrial data, data may be missing due to sensor failure or data collection problems.

[0035] Furthermore, the data with missing values ​​under normal working conditions are used as training data and integrated into the original measurement matrix.

[0036] In the present invention, the data with missing values ​​under normal working conditions is used as training data and integrated into the original measurement matrix, which can ensure the representativeness of the training data, improve the accuracy of missing value filling, reduce the interference of abnormal data, establish an effective benchmark, and enhance the adaptability of the data processing method. These benefits jointly promote the overall performance and accuracy of the data preprocessing and anomaly detection system.

[0037] S2: Fill in the missing values ​​in the training data.

[0038] Specifically, the mean and variance of the remaining complete data in the original measurement matrix are calculated, and according to the mean and variance of the remaining complete data in the original measurement matrix, the original measurement matrix is ​​simply filled into a matrix without missing values. The remaining complete data refers to the data part without missing values ​​in the original measurement matrix.

[0039] In the present invention, by calculating the mean and variance of the remaining complete data in the original measurement matrix and filling with these statistics, it helps to improve the integrity of the data, simplify the data processing process, maintain the statistical characteristics of the data, reduce the deviation in data processing, support subsequent modeling and analysis, and improve the availability of data. These benefits jointly promote the efficiency of data processing and the performance of the model.

[0040] S3: Determine the optimal solution for the padded training data by minimizing the detection metric.

[0041] In the present invention, the method of minimizing the detection index can ensure that the data after filling processing is closer to the real data. This means that the data matrix after filling can better reflect the actual measurement situation, thereby improving the accuracy of the data. By minimizing the detection index, the errors and deviations introduced by inaccurate data filling can be effectively reduced. The detection index is usually used to evaluate the quality of data filling, and minimizing the detection index helps to find the filling scheme with the smallest error.

[0042] In a possible implementation, S3 specifically includes sub-steps S301 to S307:

[0043] S301: Construct the first single-step sliding window according to the training data after padding processing:

[0044]

[0045]

[0046] Among them, X krepresents the measurement matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, represents the original measurement matrix, x i,j It represents the jth measurement variable in the i-th sample in the original measurement matrix, that is, the training data after filling processing, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables.

[0047] In the present invention, the sliding window technology allows for localized analysis of data. Through the sliding window construction method, data in a specific time period can be focused on, thereby capturing local features and patterns in the data, which helps to improve the sensitivity and adaptability of the model to local changes in the data. The sliding window technology can help divide the data into multiple windows, thereby reducing the amount of data processed each time. This helps to optimize computing efficiency, avoid full processing of the entire data set, and improve the speed and efficiency of data processing.

[0048] S302: Standardize the data in the first single-step sliding window:

[0049]

[0050] in, represents the normalized measurement matrix at time k, represents the sample mean, Represents the sample standard deviation.

[0051] It should be noted that X k is a matrix with w rows and m columns, μ 0 The m*1 column vector is the sample mean of each column in the matrix, and 1 is a w*1 matrix full of 1s to implement the mean subtraction operation on each row of the matrix.

[0052] It should be noted that the sample mean and sample standard deviation are obtained by statistically calculating a large amount of training data under normal working conditions.

[0053] In the present invention, the standardization process eliminates the dimensional differences between different features by converting the data into a standard normal distribution with a mean of 0 and a variance of 1. This allows the data to be compared on the same scale, avoiding the deviation of the analysis results caused by different dimensions. The standardization process ensures the consistency of the data in different time windows, so that the data in each window has the same statistical properties. This helps to maintain the consistency of the data processing process, thereby improving the reliability of analysis and modeling.

[0054] S303: Calculate the covariance matrix of the standardized measurement matrix:

[0055]

[0056] Among them, cov represents the covariance matrix, C k represents the sample covariance matrix of the standardized measurement matrix at time k, T represents the matrix transpose operation, Represents the first diagonal matrix, where the element values ​​in the first diagonal matrix are C k The eigenvalues ​​of , where the eigenvalues ​​are arranged in descending order, Represents C k The eigenvectors corresponding to each eigenvalue of .

[0057] In the present invention, the calculation of the covariance matrix can help identify outliers in the data. If some data points show abnormal patterns in the covariance matrix, it may indicate that these data points are abnormal or have problems. Understanding the covariance matrix can help optimize model design and selection. For example, in high-dimensional data, by calculating the covariance matrix and selecting important features, the prediction accuracy of the model can be improved and overfitting can be reduced.

[0058] S304: Transform the standardized measurement matrix:

[0059]

[0060] in, Represents the first variable at time k, that is, the linear combination of the standardized measurement vectors in the first single-step sliding window.

[0061] In the present invention, by performing linear transformation on the standardized measurement matrix, the main features or principal components can be extracted. The transformed data can better reveal the inherent structure of the data, retain the most important change information, and reduce redundant data.

[0062] S305: By optimizing the missing values ​​in the original measurement matrix through a minimization objective function constructed with the minimization of the detection index as the goal, the optimal solution under each window in the first single-step sliding window is determined:

[0063]

[0064] Among them, min means taking the minimum value, represents the normalized measurement matrix at time k, represents the λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the covariance matrix of the training data sample, λ represents the operation of finding the eigenvalue of the covariance matrix of the sample in the current window, represents the λ obtained from the training data * The mean of represents μ obtained from training data * The sample standard deviation of represents μ obtained from training data * The mean value, μ * Represents the mean of all first variables calculated from the training data.

[0065] In the present invention, by minimizing the objective function, the parameters that make the missing values ​​fill in the best can be found. This helps to generate a more accurate complete data matrix, reduce errors in the filling process, and thus improve data quality. By optimizing the filling of missing values, it can be ensured that the data used by the model during training is more accurate. This will improve the predictive performance and stability of the model and avoid errors introduced by inaccurate filling of missing values.

[0066] In a possible implementation, S305 specifically includes sub-steps S3051 and S3052:

[0067] S3051: Convert the minimization objective function into the standard ADMM form to obtain the standardized function:

[0068]

[0069] in, represents the normalized measurement matrix at time k, f 1 Indicates that T k Related functions, T k represents the variable of the kth window, f 2 Represents the same as C k Related functions, C k represents the sample covariance matrix of the kth window, g 1 represents the indicator function, when x ij When missing, otherwise x ij represents the value of the i-th row and j-th column of the k-th window, represents the indicator function for the location of missing values, represents the intermediate variable associated with the normalized measurement matrix of the kth window, g 2 (P k ) represents the k The indicator function, P k Represents the eigenvector of the sample covariance matrix of the kth window.

[0070] Among them, the Alternating Direction Method of Multipliers (ADMM) is an algorithm for solving optimization problems, especially for large-scale optimization problems with constraints. ADMM decomposes the original problem into more manageable sub-problems and optimizes these sub-problems alternately to find the optimal solution. It combines the advantages of the Lagrange multiplier method and the coordinate descent method, and by introducing additional variables and optimization steps, it converts complex optimization problems into simpler sub-problems for solution, thereby improving computational efficiency and enhancing the flexibility of the algorithm. ADMM is widely used in fields such as machine learning, signal processing, and statistics.

[0071] S3052: Iteratively update the normalization function to determine the optimal solution in each window:

[0072]

[0073] Among them, T [k+1] represents the variable in the k+1th iteration, ρ 1 represents the first penalty coefficient, represents the matrix in the current k-th iteration, P [k] represents the eigenvector of the sample covariance matrix after the kth iteration, U [k] represents the first intermediate variable used to satisfy the constraints, Indicates the calculation of the square of the F norm of the matrix, C [k+1] represents the sample covariance matrix in the k+1th iteration, ρ 2 represents the second penalty coefficient, represents the normalized measurement matrix in the kth iteration, V [k] represents the second intermediate variable used to satisfy the constraint in the kth iteration, represents the measurement matrix after normalization in the k+1th iteration, ρ 3 Represents the third penalty coefficient, W [k] represents the third intermediate variable used to satisfy the constraints, represents the matrix in the k+1th iteration, P [k+1] represents the eigenvector of the sample covariance matrix calculated by the measurement matrix after the k+1th iteration, U [k+1] represents the first intermediate variable updated after the k+1th iteration, V [k +1] represents the second intermediate variable in the k+1th iteration, W [k+1] represents the third intermediate variable after the k+1th iteration.

[0074] In the present invention, by converting the minimization objective function into a standard ADMM form and performing iterative updates, the optimization efficiency and solution accuracy can be effectively improved, complex optimization problems can be handled, and good convergence and stability can be provided. This process helps find the optimal solution in the problem of filling missing values, ensures the accuracy of data and the reliability of analysis, and can handle large-scale data and complex constraints, improving the quality of overall data processing and analysis.

[0075] In a possible implementation, S3052 specifically includes sub-steps S3052A to S3052I:

[0076] S3052A: Initialize the first penalty coefficient ρ 1 , the second penalty coefficient ρ 2 , the third penalty coefficient ρ 3 , the first intermediate variable U, the second intermediate variable V, the third intermediate variable W and the convergence control limit ε 0 .

[0077] S3052B: Based on the training data after filling processing, obtain the first variable, the sample covariance matrix, the measurement matrix of the current window after standardization processing, the eigenvector of the sample covariance matrix and the iterative initial value of the intermediate variable related to the measurement matrix after standardization processing.

[0078] S3052C: Update the first variable:

[0079]

[0080] Where γ represents the first iteration intermediate variable used to complete the calculation in the kth iteration, γ i represents the i-th value of γ, b i represents the i-th column of matrix B, Represents the training data The mean of Represents the training data The sample standard deviation of represents the mean of the i-th column of the first variable, B [k] Represents the second iteration intermediate variable used to complete the calculation in the kth iteration.

[0081] It should be noted that γ is an m-dimensional column vector. To ensure that the w rows of U are all minus the transpose of γ, it is multiplied by a w-dimensional column vector.

[0082] S3052D: Update sample covariance matrix:

[0083] C [k+1] =Q [k] Λ * Q [k]T←

[0084]

[0085] Among them, Q [k] express The corresponding eigenvector, represents the value of the current window measurement matrix after normalization at the kth iteration, Λ * (C) represents the diagonal matrix composed of the eigenvalues ​​of the sample covariance matrix in the k+1th iteration, Represents Λ * (C) The value of the diagonal, λ i Represents the operation of calculating the eigenvalue, and obtains the i-th eigenvalue, represents the mean of the eigenvalues ​​calculated from the training data, Represents the sample standard deviation of the eigenvalues ​​calculated from the training data.

[0086] S3052E: Order Initialize the convergence control limit ε X , iteratively update the original measurement matrix until the first preset update condition is reached and the update stops:

[0087]

[0088] in, represents the decision variable at step t in iteration k. , Y t and Y t+1 represents the intermediate variable, C [k+1] represents the sample covariance matrix in the k+1th iteration, V [k+1] represents the variable in the k+1th iteration, I represents the unit matrix, ω represents the weight coefficient, represents the first preset update condition, represents the decision variable at step t+1 in iteration k.

[0089] S3052F: When When , the standardized measurement matrix is ​​updated, that is, the missing position corresponding to the original measurement matrix is ​​updated

[0090]

[0091] TP=T [k+1] P [k]T ,UP=U [k] P [k]T

[0092] in, Represents an intermediate variable The value of row i and column j, TP ijrepresents the value of TP in row i and column j, UP ij represents the value of the i-th row and j-th column of UP, Represents the value of the i-th row and j-th column of the measurement matrix after the k+1th iteration of the current window, Represents the value of W in the i-th row and j-th column after the k-th iteration.

[0093] S3052G: Update the eigenvectors of the sample covariance matrix:

[0094]

[0095] F [k] =T [k+1] +U [k]

[0096] Among them, P [k+1]1 represents the eigenvector of the sample covariance matrix at the k+1th iteration, F [k] Represents the third iteration intermediate variable used to complete the calculation in the kth iteration.

[0097] S3052H: Update the first intermediate variable, the second intermediate variable, and the third intermediate variable.

[0098] S3052I: Repeat S3052C to S3052H until the second preset update condition is reached and the update is stopped to determine the optimal solution in each window:

[0099] O [k+1] -O [k] <ε 0

[0100] O [k] =f 1 (T [k] )+f 2 (C [k] )

[0101] Among them, O [k+1] -O [k] <ε 0 represents the second preset update condition, O [k+1] represents the value of the objective function after the k+1th iteration, O [k] represents the value of the objective function after the kth iteration, T [k] represents the variable after the kth iteration.

[0102] In the present invention, the ADMM method performs well in processing large-scale data. Through iterative updating, large-scale optimization problems can be efficiently solved, which is suitable for missing value filling tasks of complex data. By iteratively updating the minimization objective function in the form of standard ADMM, the missing value filling problem can be solved efficiently and accurately. The initialization step ensures the stability and convergence of the algorithm, and the iterative update process improves the calculation accuracy and result stability, while being able to handle complex optimization problems and adapt to data changes. This method not only improves the accuracy of the filled data, but also makes the entire optimization process more efficient and flexible.

[0103] S306: Take the average of the optimal solutions in each window to obtain the optimal solution of the training data after padding processing under the current initialization conditions.

[0104] S307: Repeat S301 to S306 until the third preset update condition is reached and the update is stopped, and the optimal solution of the measurement matrix is ​​obtained as the optimal matrix:

[0105] ‖X l+1 -X l || 2 <ε 1

[0106] Among them, ε 1 represents the third preset update condition, X l+1 represents the measurement matrix after the l+1th iteration, X l represents the measurement matrix after the lth iteration, |||| 2 It means to find the bi-norm of a matrix.

[0107] It should be noted that the optimal solution of the training data after the padding process is the optimal matrix.

[0108] In the present invention, by averaging the optimal solution of each window, the noise caused by local data fluctuations can be reduced, making the optimal solution finally obtained smoother and more stable. This helps to improve the reliability of the filling result. By using a structure similar to the EM algorithm, the process of calculating statistics, minimizing the detection index and updating the measurement matrix is ​​alternately performed, so that the training set tends to be stable during the update.

[0109] S4: Calculate the statistics of the variables based on the optimal solution of the training data after the padding process.

[0110] In a possible implementation, S4 specifically includes sub-steps S401 to S405:

[0111] S401: Construct a second single-step sliding window according to the optimal solution of the training data after padding processing:

[0112]

[0113] Among them, X' k represents the optimal matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, X' represents the optimal matrix, x i ' ,j It represents the optimal solution of the jth measurement variable in the i-th sample in the optimal matrix, that is, the training data after filling processing, 1≤i≤n,1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables.

[0114] S402: Standardize the data in the second single-step sliding window:

[0115]

[0116] in, represents the optimal matrix after normalization at time k, μ 0 represents the sample mean, Σ 0 Represents the sample standard deviation.

[0117] It should be noted that X' k is a matrix with w rows and m columns, μ 0 The m*1 column vector is the sample mean of each column in the matrix, and 1 is a w*1 matrix full of 1s to implement the mean subtraction operation on each row of the matrix.

[0118] S403: Calculate the covariance matrix of the optimal matrix after standardization:

[0119]

[0120] Among them, cov represents the covariance matrix, C' k represents the sample covariance matrix of the optimal matrix at time k after normalization, T represents the matrix transposition operation, Λ' k Represents the second diagonal matrix, where the element value in the second diagonal matrix is ​​C' k The eigenvalues ​​of , where the eigenvalues ​​are arranged in descending order, P k ' means C' k The eigenvectors corresponding to each eigenvalue of .

[0121] S404: Transform the optimal matrix after normalization:

[0122]

[0123] Among them, T' k Represents the second variable at time k, that is, the linear combination of the standardized measurement vectors in the second single-step sliding window.

[0124] S405: In the second single-step sliding window, the statistical information of the second variable is calculated to obtain the statistical value of the second variable:

[0125] Θ T =[μ T |V T |Γ T ]

[0126]

[0127] Where μ represents the sample mean including the second variable The first-order statistic of , V represents the sample variance including the second variable The second-order statistic of , Γ represents the sample skewness including the second variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the second variable, j represents an intermediate variable that traverses every row of the i-th variable, represents the standard deviation of the i-th column of the second variable, Represents the sample mean of the i-th variable in the second variable under normal operating conditions.

[0128] In the present invention, by calculating the statistics of the variables (such as mean, variance, skewness and kurtosis), the quality of the data after filling processing can be evaluated to ensure the accuracy and reliability of the filling results. This helps to determine whether the filled data conforms to the statistical characteristics of the actual data. Statistics (mean, variance, skewness, kurtosis, etc.) can reveal the distribution characteristics and trends of the data. This helps to understand the basic characteristics of the data, such as the central tendency, dispersion and distribution form of the data, thereby providing a basis for subsequent analysis.

[0129] S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level.

[0130] In a possible implementation, S5 specifically includes sub-steps S501 and S502:

[0131] S501: Calculate the detection index according to the statistics of the second variable:

[0132]

[0133] Among them, D k represents the detection index at time k, represents the statistics of the variables in the kth sliding window, represents the reference mean of Θ obtained from historical data under normal working conditions, s represents the number of types of selected statistics, || || p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

[0134] S502: Use empirical methods to determine control limits using historical data and confidence levels under normal operating conditions.

[0135] It should be noted that the confidence level is also called the significance level.

[0136] For example, if the significance level is chosen to be 0.01, the corresponding control limits can be obtained by making it exceed the detection index of 99% of the training data.

[0137] In the present invention, determining control limits can help identify potential outliers or abnormal patterns in training data. By setting reasonable control limits, normal data and abnormal data can be effectively distinguished, so that problems can be discovered early in industrial processes. Determining control limits can help identify potential outliers or abnormal patterns in training data. By setting reasonable control limits, normal data and abnormal data can be effectively distinguished, so that problems can be discovered early in industrial processes. Using empirical methods and confidence levels to determine control limits, based on statistical principles, ensures that the setting of control limits is scientific and reasonable, and enhances the statistical control capability of the system.

[0138] S6: Obtain real-time data with missing values.

[0139] Furthermore, the acquired real-time data containing missing values ​​are integrated into a real-time matrix.

[0140] In the present invention, by acquiring and integrating real-time data, the industrial process can be monitored and analyzed in real time, abnormal situations can be discovered and responded to in a timely manner, and the stability and safety of the process can be ensured.

[0141] S7: Fill in the missing values ​​in the real-time data.

[0142] Specifically, according to the mean and variance of the training data, the real-time matrix is ​​simply filled into a matrix without missing values.

[0143] In the present invention, by filling missing values, the integrity of the real-time data set can be ensured so that all data points have values, thereby avoiding data bias caused by missing values ​​during analysis and processing. Filling missing values ​​can reduce gaps or inconsistencies in the data set, thereby improving data quality and making the data more accurate and representative.

[0144] S8: Determine the optimal solution for the complete data after filling by minimizing the detection index.

[0145] In a possible implementation, S8 specifically includes sub-steps S801 to S805:

[0146] S801: Standardize the complete data after filling:

[0147]

[0148] in, represents the real-time matrix after normalization, μ 0 represents the sample mean, Σ 0 represents the sample standard deviation, X" represents the real-time matrix, x i ” ,j It represents the jth measured variable in the i-th sample in the real-time matrix, that is, the complete data after filling processing, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measured variables.

[0149] It should be noted that X” is a matrix with w rows and m columns, μ 0 The m*1 column vector is the sample mean of each column in the matrix, and 1 is a w*1 matrix with all 1s to perform the mean subtraction operation on each row of the matrix.

[0150] S802: Calculate the covariance matrix of the normalized real-time matrix:

[0151]

[0152] Among them, cov represents the covariance matrix, C” represents the sample covariance matrix of the real-time matrix after standardization, T represents the matrix transpose operation, Λ” represents the second diagonal matrix, the element values ​​in the second diagonal matrix are the eigenvalues ​​of C” and are arranged in descending order, and P” represents the eigenvector corresponding to the eigenvalue of C”.

[0153] S803: Transform the real-time matrix after the standardization process:

[0154]

[0155] Wherein, "T" represents the third variable.

[0156] S804: By optimizing the missing values ​​in the real-time matrix through the minimization objective function constructed with the minimization of the detection index as the goal, the optimal solution under each window is determined:

[0157]

[0158] Among them, min means taking the minimum value, represents the real-time matrix after normalization, represents the λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the sample covariance matrix calculated from the training data, λ represents the operation of finding the eigenvalue, represents the λ obtained from the training data* The mean of represents μ obtained from training data * The sample standard deviation of represents μ obtained from training data * The mean value, μ * Represents the mean of the variables calculated from the training data.

[0159] S805: Take the average value of the optimal solution in each window to obtain the optimal solution of the complete data after the filling process.

[0160] In the present invention, by minimizing the detection index, the filling result can be further optimized when processing the filled data, making the data more accurate and reducing the error introduced by filling. Standardization and covariance matrix calculation can ensure that the data is processed and analyzed under a unified standard, ensure data consistency, and avoid data inconsistency caused by different processing methods. The optimal solution under each window is averaged to ensure that all processed data are consistent and comparable, reducing data deviation.

[0161] S9: Calculate the statistics of the variables based on the optimal solution of the complete data after the filling process.

[0162] In a possible implementation, S9 specifically includes:

[0163] The optimal solution of the complete data after filling is input into the RTCSA model, and the statistical information of the third variable is calculated to obtain the statistics of the third variable:

[0164] Θ' T =[μ' T |V' T |Γ' T ]

[0165]

[0166] Where μ' represents the sample mean including the third variable The first-order statistic of , V' represents the sample variance including the third variable The second-order statistic of , Γ' represents the sample skewness including the third variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the third variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the ith column of the third variable, It represents the sample mean of the i-th variable in the third variable under normal working conditions, which is calculated from the training data.

[0167] Among them, the Recursive Transformed Component Statistical Analysis (RTCSA) model is a multivariate statistical process monitoring method for early fault detection problems. The model obtains orthogonal variables by constructing a single-step sliding window. The statistical information of the variables can reflect the important characteristics of the process, thereby realizing the detection of faults. The RTCSA model can calculate the statistics of the data in real time, and use these statistics to compare with the data under normal conditions, so as to promptly discover possible problems or deviations in the system. It is widely used in industrial process monitoring, data analysis and other fields to ensure the stability and reliability of the system.

[0168] In the present invention, through the RTCSA (Recursive Variable Statistical Analysis) model, a comprehensive statistical analysis can be performed on the filled data, considering the sample mean, variance, skewness and kurtosis and other statistics, to provide a more accurate and comprehensive data description. The optimal solution of the complete data after filling is used as input to ensure that the statistical calculation is based on the most reliable data, thereby improving the reliability of statistical analysis.

[0169] S10: Calculate the detection index according to the statistics of the variables of the real-time data.

[0170] In a possible implementation manner, S10 specifically includes:

[0171] According to the statistics of the third variable, the detection index is calculated:

[0172]

[0173] Among them, D represents the detection index, Θ represents the statistics of the variables in the sliding window, Θ 0 represents the reference mean of Θ obtained from historical data under normal working conditions, s represents the number of types of selected statistics, |||| p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

[0174] In the present invention, the detection index is calculated based on the statistics of the variables of real-time data, which can provide important functions such as real-time monitoring, anomaly detection, data quality control and decision support. It helps to improve the reliability, responsiveness and intelligence level of the system, and supports the optimization and early warning mechanism of the system. This method ensures the accurate analysis and management of real-time data, and provides users with more effective data processing and decision support.

[0175] S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

[0176] Specifically, determine whether the detection index is greater than the control limit. If D>δk , δ k Indicates the control limit, that is, when the detection index is greater than the control limit, it is judged as an abnormal condition, otherwise it is judged as a normal condition.

[0177] In the present invention, by setting control limits, clear thresholds can be established to distinguish normal working conditions from abnormal working conditions, making the abnormality detection process more structured and systematic. By comparing the detection index with the control limit, maintenance work can be effectively guided, and those abnormal conditions that exceed the control limit can be handled preferentially, thereby optimizing the allocation of maintenance resources.

[0178] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0179] In the present invention, the optimal solution of the filled training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated according to the optimal solution of the filled training data, and the control limit is determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the filled complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set.

[0180] Reference Manual Attached Figure 2 , showing a structural schematic diagram of an industrial process fault detection system for missing data provided by the present invention.

[0181] The present invention further provides an industrial process fault detection system 30 for missing data, comprising: a memory 303 and one or more processors 301 .

[0182] One or more application programs are stored in the memory 303 , and the one or more application programs are suitable for being executed by the one or more processors 301 to implement the industrial process fault detection method for missing data described in the method embodiment.

[0183] The industrial process fault detection system 30 for missing data includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302.

[0184] The structure of the industrial process fault detection system 30 for missing data does not constitute a limitation on the embodiment of the present invention.

[0185] Processor 301 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0186] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI bus or an EISA bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0187] The memory 303 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0188] It should be noted that the industrial process fault detection system 30 for missing data can implement the above-mentioned industrial process fault detection method for missing data, and can achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0189] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0190] In the present invention, the optimal solution of the filled training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated according to the optimal solution of the filled training data, and the control limit is determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the filled complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set.

[0191] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be loaded and executed by a processor to implement the industrial process fault detection method for missing data as described in the first aspect.

[0192] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0193] There are a few points to note:

[0194] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0195] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0196] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0197] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for industrial process fault detection with missing data, characterized in that: include: S1: Obtain data with missing values ​​under normal working conditions as training data; S2: Filling missing values ​​in the training data; S3: Determine the optimal solution for the padded training data by minimizing the detection index; S4: Calculate the statistics of the variables according to the optimal solution of the training data after filling; S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level; S6: Obtain real-time data with missing values; S7: Filling missing values ​​in the real-time data; S8: Determine the optimal solution for the complete data after filling by minimizing the detection index; S9: Calculate the statistics of the variables according to the optimal solution of the complete data after filling processing; S10: Calculate the detection index according to the statistics of the variables of the real-time data; S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

2. The industrial process fault detection method for missing data according to claim 2, characterized in that: The S3 specifically includes: S301: Construct the first single-step sliding window according to the training data after padding processing: Among them, X k represents the measurement matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, represents the original measurement matrix, x i,j represents the jth measurement variable in the i-th sample in the original measurement matrix, i.e., the training data after filling, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables; S302: Standardize the data in the first single-step sliding window: in, represents the normalized measurement matrix at time k, represents the sample mean, represents the sample standard deviation; S303: Calculate the covariance matrix of the standardized measurement matrix: Among them, cov represents the covariance matrix, C k represents the sample covariance matrix of the standardized measurement matrix at time k, T represents the matrix transpose operation, represents the first diagonal matrix, where the element values ​​in the first diagonal matrix are C k , wherein the eigenvalues ​​are arranged in descending order, Represents C k The eigenvectors corresponding to each eigenvalue of ; S304: Transform the standardized measurement matrix: in, represents the first variable at time k, i.e., the linear combination of the normalized measurement vectors in the first single-step sliding window; S305: Optimizing the missing values ​​in the original measurement matrix by minimizing the objective function constructed with minimizing the detection index as the goal, and determining the optimal solution for each window in the first single-step sliding window: Among them, min means taking the minimum value, represents the normalized measurement matrix at time k, represents the λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the covariance matrix of the training data sample, λ represents the operation of finding the eigenvalue of the covariance matrix of the sample in the current window, represents the λ obtained from the training data * The mean of represents μ obtained from training data * The sample standard deviation of represents μ obtained from training data * The mean value, μ * represents the mean of all first variables calculated from the training data; S306: taking the average of the optimal solutions in each window to obtain the optimal solution of the training data after padding processing under the current initialization conditions; S307: Repeat S301 to S306 until the third preset update condition is reached and the update is stopped, and the optimal solution of the measurement matrix is ​​obtained as the optimal matrix: ||X l+1 -X l ||2<ε1 Among them, ε1 represents the third preset update condition, X l+1 represents the measurement matrix after the l+1th iteration, X l represents the measurement matrix after the lth iteration, and || ||2 represents the calculation of the matrix norm.

3. The industrial process fault detection method for missing data according to claim 2, characterized in that: The S305 specifically includes: S3051: Convert the minimization objective function into a standard ADMM form to obtain a standardized function: in, represents the standardized measurement matrix at time k, and f1 represents the k Related functions, T k represents the variable of the kth window, f2 represents the variable corresponding to C k Related functions, C k represents the sample covariance matrix of the kth window, g1 represents the indicator function, when x ij When missing, otherwise x ij represents the value of the i-th row and j-th column of the k-th window, represents the indicator function for the location of missing values, represents the intermediate variable related to the measurement matrix after the k-th window is standardized, g2(P k ) represents the k The indicator function, P k The eigenvector representing the sample covariance matrix of the kth window; S3052: Iteratively update the standardization function to determine the optimal solution in each window: Among them, T [k+1] represents the variable in the k+1th iteration, ρ1 represents the first penalty coefficient, represents the matrix in the current k-th iteration, P [k] represents the eigenvector of the sample covariance matrix after the kth iteration, U [k] represents the first intermediate variable used to satisfy the constraints, Indicates the calculation of the square of the F norm of the matrix, C [k+1] represents the sample covariance matrix in the k+1th iteration, ρ2 represents the second penalty coefficient, represents the normalized measurement matrix in the kth iteration, V [k] represents the second intermediate variable used to satisfy the constraint in the kth iteration, represents the measurement matrix after normalization in the k+1th iteration, ρ3 represents the third penalty coefficient, W [k] represents the third intermediate variable used to satisfy the constraints, represents the matrix in the k+1th iteration, P [k+1] represents the eigenvector of the sample covariance matrix calculated by the measurement matrix after the k+1th iteration, U [k+1] represents the first intermediate variable updated after the k+1th iteration, V [k+1] represents the second intermediate variable in the k+1th iteration, W [k+1] represents the third intermediate variable after the k+1th iteration.

4. The industrial process fault detection method for missing data according to claim 3, characterized in that: The S3052 specifically includes: S3052A: Initialize the first penalty coefficient ρ1, the second penalty coefficient ρ2, the third penalty coefficient ρ3, the first intermediate variable U, the second intermediate variable V, the third intermediate variable W, and the convergence control limit ε0; S3052B: Obtaining, according to the training data after filling processing, the first variable, the sample covariance matrix, the measurement matrix of the current window after standardization processing, the eigenvector of the sample covariance matrix, and the iterative initial value of the intermediate variable related to the measurement matrix after standardization processing; S3052C: Update the first variable: Where γ represents the first iteration intermediate variable used to complete the calculation in the kth iteration, γ i represents the i-th value of γ, b i represents the i-th column of matrix B, Represents the training data The mean of Represents the training data The sample standard deviation of represents the mean of the i-th column of the first variable, B [k] represents the second iteration intermediate variable used to complete the calculation in the kth iteration; S3052D: Update the sample covariance matrix: Among them, Q [k] express The corresponding eigenvector, represents the value of the current window measurement matrix after normalization at the kth iteration, Λ * (C) represents the diagonal matrix composed of the eigenvalues ​​of the sample covariance matrix in the k+1th iteration, Represents Λ * (C) The value of the diagonal, λ i Represents the operation of calculating the eigenvalue, and obtains the i-th eigenvalue, represents the mean of the eigenvalues ​​calculated from the training data, represents the sample standard deviation of the eigenvalues ​​calculated from the training data; S3052E: Order Initialize the convergence control limit ε X , iteratively update the original measurement matrix until the first preset update condition is reached and then stop updating: in, represents the decision variable at step t in iteration k. , Y t and Y t+1 represents the intermediate variable, C [k+1] represents the sample covariance matrix in the k+1th iteration, V [k+1] represents the variable in the k+1th iteration, I represents the unit matrix, ω represents the weight coefficient, represents the first preset update condition, represents the decision variable at step t+1 in the kth iteration; S3052F: When When , the standardized measurement matrix is ​​updated, that is, the missing position corresponding to the original measurement matrix is ​​updated TP=T [k+1] P [k]T ,UP=U [k] P [k]T in, Represents an intermediate variable The value of row i and column j, TP ij represents the value of TP in row i and column j, UP ij represents the value of the i-th row and j-th column of UP, Represents the value of the i-th row and j-th column of the measurement matrix after the k+1th iteration of the current window, represents the value of W in row i and column j after the kth iteration; S3052G: Update the eigenvectors of the sample covariance matrix: F [k] =T [k+1] +U [k] Among them, P [k+1] represents the eigenvector of the sample covariance matrix at the k+1th iteration, F [k] represents the third iteration intermediate variable used to complete the calculation in the kth iteration; S3052H: Update the first intermediate variable, the second intermediate variable, and the third intermediate variable; S3052I: Repeat S3052C to S3052H until the second preset update condition is reached and the update is stopped to determine the optimal solution in each window: O [k+1] -O [k] <ε0 O [k] =f1(T [k] )+f2(C [k] ) Among them, O [k+1] -O [k] <ε0 represents the second preset update condition, O [k+1] represents the value of the objective function after the k+1th iteration, O [k] represents the value of the objective function after the kth iteration, T [k] represents the variable after the kth iteration.

5. The industrial process fault detection method for missing data according to claim 2, characterized in that: The S4 specifically includes: S401: Construct a second single-step sliding window according to the optimal solution of the training data after padding processing: Among them, X' k represents the optimal matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, X' represents the optimal matrix, x' i,j represents the optimal solution of the jth measurement variable in the i-th sample in the optimal matrix, i.e. the training data after filling, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables; S402: performing standardization processing on the data in the second single-step sliding window: in, represents the optimal matrix after normalization at time k, μ0 represents the sample mean, and Σ0 represents the sample standard deviation; S403: Calculate the covariance matrix of the optimal matrix after standardization: Among them, cov represents the covariance matrix, C' k represents the sample covariance matrix of the optimal matrix at time k after normalization, T represents the matrix transpose operation, Λ' k represents the second diagonal matrix, where the element value in the second diagonal matrix is ​​C' k The eigenvalues ​​of , wherein the eigenvalues ​​are arranged in descending order, P' k Represents C' k The eigenvectors corresponding to each eigenvalue of ; S404: Transform the optimal matrix after normalization: Among them, T' k represents the second variable at time k, i.e., the linear combination of the normalized measurement vectors in the second single-step sliding window; S405: In the second single-step sliding window, calculate the statistical information of the second variable to obtain the statistical value of the second variable: I T =[μ T |V T |C T ] Where μ represents the sample mean including the second variable The first-order statistic of , V represents the sample variance including the second variable The second-order statistic of , Γ represents the sample skewness including the second variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the second variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the i-th column of the second variable, Represents the sample mean of the i-th variable in the second variable under normal operating conditions.

6. The industrial process fault detection method for missing data according to claim 5, characterized in that: The S5 specifically includes: S501: Calculate the detection index according to the statistics of the second variable: Among them, D k represents the detection index at time k, represents the statistics of the variables in the kth sliding window, represents the reference mean of Θ obtained from historical data under normal operating conditions, s represents the number of types of selected statistics, || || p represents the vector p-norm, represents the sample standard deviation of the variable statistic; S502: Use empirical methods to determine control limits using historical data and confidence levels under normal operating conditions.

7. The industrial process fault detection method for missing data according to claim 1, characterized in that: S8 specifically includes: S801: Standardize the complete data after filling: in, represents the real-time matrix after standardization, μ0 represents the sample mean, Σ0 represents the sample standard deviation, X” represents the real-time matrix, x” i,j It represents the jth measured variable in the i-th sample in the real-time matrix, i.e., the complete data after filling processing, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measured variables; S802: Calculate the covariance matrix of the normalized real-time matrix: Among them, cov represents the covariance matrix, C" represents the sample covariance matrix of the real-time matrix after standardization, T represents the matrix transpose operation, Λ” represents the second diagonal matrix, the element values ​​in the second diagonal matrix are the eigenvalues ​​of C” and are arranged in descending order, and P” represents the eigenvector corresponding to the eigenvalue of C”; S803: Transform the real-time matrix after the standardization process: Wherein, T” represents the third variable; S804: Optimizing the missing values ​​in the real-time matrix by minimizing the objective function constructed with minimizing the detection index as the goal, and determining the optimal solution under each window: Among them, min means taking the minimum value, represents the real-time matrix after normalization, represents the λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the sample covariance matrix calculated from the training data, λ represents the operation of finding the eigenvalue, represents the λ obtained from the training data * The mean of represents μ obtained from training data * The sample standard deviation of represents μ obtained from training data * The mean value, μ * represents the mean of the variables calculated from the training data; S805: Take the average value of the optimal solution in each window to obtain the optimal solution of the complete data after the filling process.

8. The industrial process fault detection method for missing data according to claim 7, characterized in that: The S9 is specifically: The optimal solution of the complete data after filling is input into the RTCSA model, and the statistical information of the third variable is calculated to obtain the statistics of the third variable: I' T =[m' T |V' T |C' T ] Where μ' represents the sample mean including the third variable The first-order statistic of , V' represents the sample variance including the third variable The second-order statistic of , Γ' represents the sample skewness including the third variable and sample kurtosis Higher-order statistics of , t' i represents the i-th column of the third variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the ith column of the third variable, It represents the sample mean of the i-th variable in the third variable under normal working conditions, which is calculated from the training data.

9. The industrial process fault detection method for missing data according to claim 8, characterized in that: The S10 is specifically: According to the statistics of the third variable, the detection index is calculated: Where D represents the detection index, Θ represents the statistic of the variable in the sliding window, Θ0 represents the reference mean of Θ obtained from the historical data under normal working conditions, s represents the number of types of selected statistics, || || p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

10. An industrial process fault detection system for missing data, characterized in that: include: memory and one or more processors; One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the industrial process fault detection method for missing data according to any one of claims 1 to 9.

Images