Industrial process fault detection method and system for missing data

Abstract

The present invention provides an industrial process fault detection method and system for missing data, relating to the technical field of industrial process fault detection. The method comprises: obtaining data containing missing values ​​under normal operating conditions as training data; filling the missing values ​​in the training data; determining the optimal solution of the filled training data by minimizing a detection index; calculating the statistics of the variables based on the optimal solution of the filled training data; determining control limits based on the statistics of the variables of the training data and the confidence level; obtaining real-time data containing missing values; filling the missing values ​​in the real-time data; determining the optimal solution of the filled complete data by minimizing the detection index; calculating the statistics of the variables based on the optimal solution of the filled complete data; calculating the detection index based on the statistics of the variables of the real-time data; and determining an abnormal operating condition when the detection index is greater than the control limit.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial process fault detection, and in particular to an industrial process fault detection method and system for missing data. Background Art

[0002] With economic development and the continuous expansion of industrial production, ensuring production process safety and controlling product quality have become significant challenges. In actual production processes, many serious accidents often stem from minor faults. Early detection of these faults is crucial to ensuring the normal operation of the production process and improving product quality. During the production process, a large number of sensors are responsible for measuring various process variables, and their readings directly reflect the state of the production process. However, over long-term operation, sensor accuracy degrades, potentially causing drift, deviation, or complete failure, leading to data loss and anomalies. Furthermore, data loss can occur during data entry and storage. These factors often result in varying degrees of missing data in the acquired production data. Therefore, research on early fault detection for missing data is crucial for improving the automation level of production processes, reducing manual intervention, and ensuring production safety.

[0003] Current methods for dealing with missing data primarily include data recovery methods based on low-rank algorithms and other techniques. Low-rank algorithms assume that matrices have a low-rank or near-low-rank structure and recover data by finding a low-rank matrix consistent with the incomplete matrix. Other methods include inferring missing data using Bayesian statistical models and maximizing the likelihood function.

[0004] However, these methods suffer from high false alarm rates in practical applications, cannot fully restore data integrity, and cannot effectively detect early failures. Summary of the Invention

[0005] In order to solve the technical problems that traditional missing data processing methods have a high false alarm rate in practical applications, cannot fully restore data integrity, and cannot effectively detect early faults, the present invention provides an industrial process fault detection method and system for missing data.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect:

[0008] An embodiment of the present invention provides an industrial process fault detection method for missing data, comprising:

[0009] S1: Obtain data with missing values ​​under normal working conditions as training data;

[0010] S2: Filling missing values ​​in the training data;

[0011] S3: Determine the optimal solution for the padded training data by minimizing the detection metric;

[0012] S4: Calculate the statistics of the variables based on the optimal solution of the training data after filling;

[0013] S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level;

[0014] S6: Obtain real-time data with missing values;

[0015] S7: Filling missing values ​​in the real-time data;

[0016] S8: Determine the optimal solution for the complete data after filling by minimizing the detection index;

[0017] S9: Calculate the statistics of the variables based on the optimal solution of the complete data after filling;

[0018] S10: Calculate detection indicators based on the statistics of the variables of the real-time data;

[0019] S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

[0020] Second aspect:

[0021] An embodiment of the present invention provides an industrial process fault detection system for missing data, comprising: a memory and one or more processors;

[0022] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the above-mentioned industrial process fault detection method for missing data.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] In the present invention, the optimal solution of the padded training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated based on the optimal solution of the padded training data, and the control limits are determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the padded complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal operating condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flowchart of an industrial process fault detection method for missing data provided by an embodiment of the present invention;

[0027] Figure 2 A schematic structural diagram of an industrial process fault detection system for missing data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0030] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0031] Reference Manual Figure 1 , shows a flow chart of an industrial process fault detection method for missing data provided by an embodiment of the present invention.

[0032] An embodiment of the present invention provides an industrial process fault detection method for missing data. The method can be implemented by an industrial process fault detection device for missing data, which can be a terminal or a server. The processing flow of the industrial process fault detection method for missing data may include the following steps:

[0033] S1: Obtain data with missing values ​​under normal working conditions as training data.

[0034] It should be noted that normal operating conditions refer to industrial data collected under normal operating conditions. In actual industrial data, data may be missing due to sensor failure or data collection problems.

[0035] Furthermore, the data with missing values ​​under normal working conditions are used as training data and integrated into the original measurement matrix.

[0036] In this paper, data with missing values ​​under normal operating conditions is used as training data and integrated into the original measurement matrix. This ensures the representativeness of the training data, improves the accuracy of missing value filling, reduces the interference of abnormal data, establishes an effective benchmark, and enhances the adaptability of the data processing method. These benefits collectively improve the overall performance and accuracy of the data preprocessing and anomaly detection system.

[0037] S2: Fill in the missing values ​​in the training data.

[0038] Specifically, the mean and variance of the remaining complete data in the original measurement matrix are calculated. According to the mean and variance of the remaining complete data in the original measurement matrix, the original measurement matrix is ​​simply filled into a matrix without missing values. The remaining complete data refers to the data part without missing values ​​in the original measurement matrix.

[0039] In this invention, by calculating the mean and variance of the remaining complete data in the original measurement matrix and using these statistics to fill in the gaps, we can help improve data integrity, simplify the data processing process, maintain the statistical characteristics of the data, reduce bias in data processing, support subsequent modeling and analysis, and improve data usability. These benefits collectively promote data processing efficiency and model performance.

[0040] S3: Determine the optimal solution for the padded training data by minimizing the detection metric.

[0041] In the present invention, the method of minimizing the detection index can ensure that the data after the filling process is closer to the real data. This means that the filled data matrix can better reflect the actual measurement situation, thereby improving the accuracy of the data. By minimizing the detection index, the errors and deviations introduced by inaccurate data filling can be effectively reduced. The detection index is generally used to evaluate the quality of data filling, and minimizing the detection index helps to find the filling solution with the smallest error.

[0042] In a possible implementation, S3 specifically includes sub-steps S301 to S307:

[0043] S301: Construct the first single-step sliding window based on the training data after padding:

[0044]

[0045]

[0046] Among them, X kRepresents the measurement matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, represents the original measurement matrix, x i,j It represents the jth measurement variable in the i-th sample in the original measurement matrix, that is, the training data after filling processing, 1≤i≤n,1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables.

[0047] In this invention, sliding window technology allows for localized data analysis. By constructing a sliding window, we can focus on data within a specific time period, capturing local features and patterns within the data. This helps improve the model's sensitivity and adaptability to local variations in the data. Sliding window technology can help divide data into multiple windows, reducing the amount of data processed at each time. This helps optimize computational efficiency, avoids processing the entire dataset, and improves data processing speed and efficiency.

[0048] S302: Standardize the data in the first single-step sliding window:

[0049]

[0050] in, represents the normalized measurement matrix at time k, represents the sample mean, represents the sample standard deviation.

[0051] It should be noted that X k is a matrix with w rows and m columns, μ0 is an m*1 column vector, that is, the sample mean of each column in the matrix, and 1 is a w*1 matrix with all 1s to implement the mean subtraction operation on each row of the matrix.

[0052] It should be noted that the sample mean and sample standard deviation are obtained by statistical calculation of a large amount of training data under normal working conditions.

[0053] In this paper, normalization eliminates dimensional differences between features by converting data to a standard normal distribution with a mean of 0 and a variance of 1. This allows data to be compared on the same scale, avoiding bias in analytical results caused by dimensional differences. Normalization ensures data consistency across different time windows, ensuring that data in each window has the same statistical properties. This helps maintain consistency in data processing, thereby improving the reliability of analysis and modeling.

[0054] S303: Calculate the covariance matrix of the normalized measurement matrix:

[0055]

[0056] Among them, cov represents the covariance matrix, C k represents the sample covariance matrix of the standardized measurement matrix at time k, T represents the matrix transpose operation, Represents the first diagonal matrix, where the element value in the first diagonal matrix is ​​C k The eigenvalues ​​of , where the eigenvalues ​​are arranged in descending order, Indicates C k The eigenvectors corresponding to each eigenvalue of .

[0057] In the present invention, the calculation of the covariance matrix can help identify outliers in the data. If certain data points show unusual patterns in the covariance matrix, it may indicate that these data points are abnormal or have problems. Understanding the covariance matrix can help optimize model design and selection. For example, in high-dimensional data, by calculating the covariance matrix and selecting important features, the model's prediction accuracy can be improved and overfitting can be reduced.

[0058] S304: Transform the normalized measurement matrix:

[0059]

[0060] in, Represents the first variable at time k, that is, the linear combination of the normalized measurement vectors within the first single-step sliding window.

[0061] In the present invention, by performing a linear transformation on the standardized measurement matrix, the main features or principal components can be extracted. The transformed data can better reveal the inherent structure of the data, retain the most important change information, and reduce redundant data.

[0062] S305: Optimize the missing values ​​in the original measurement matrix by minimizing the objective function constructed with the goal of minimizing the detection index, and determine the optimal solution for each window in the first single-step sliding window:

[0063]

[0064] Among them, min means taking the minimum value, represents the normalized measurement matrix at time k, represents λ obtained from the training data * The sample standard deviation, λ * Represents the eigenvalue of the covariance matrix of the training data sample, λ represents the operation of finding the eigenvalue of the covariance matrix of the sample of the current window, represents λ obtained from the training data * The mean of represents μ obtained from the training data * The sample standard deviation of represents μ obtained from the training data * The mean value, μ * Represents the mean of all first variables calculated from the training data.

[0065] In this invention, by minimizing the objective function, we can find the optimal parameters for filling missing values. This helps generate a more accurate and complete data matrix, reduces errors during the filling process, and thus improves data quality. By optimizing the filling of missing values, we can ensure that the data used during model training is more accurate. This improves the model's predictive performance and stability, and avoids errors introduced by inaccurate filling of missing values.

[0066] In a possible implementation, S305 specifically includes sub-steps S3051 and S3052:

[0067] S3051: Convert the minimization objective function into the standard ADMM form to obtain the standardized function:

[0068]

[0069] in, represents the normalized measurement matrix at time k, and f1 represents the value of T k Related functions, T k represents the variable of the kth window, f2 represents the variable of C k Related functions, C k represents the sample covariance matrix of the kth window, g1 represents the indicator function, when x ij When missing, otherwise x ij represents the value of the i-th row and j-th column of the k-th window, represents the indicator function for the location of missing values, represents the intermediate variable related to the measurement matrix after the normalization of the kth window, g2(P k ) represents the value of P k The indicator function, P k Represents the eigenvector of the sample covariance matrix of the k-th window.

[0070] Among them, the Alternating Direction Method of Multipliers (ADMM) is an algorithm used to solve optimization problems, especially for large-scale optimization problems with constraints. ADMM breaks down the original problem into more manageable subproblems and alternately optimizes these subproblems to find the optimal solution. It combines the advantages of the Lagrange multiplier method and the coordinate descent method. By introducing additional variables and optimization steps, it transforms complex optimization problems into simpler subproblems for solution, thereby improving computational efficiency and enhancing algorithmic flexibility. ADMM is widely used in fields such as machine learning, signal processing, and statistics.

[0071] S3052: Iteratively update the normalization function to determine the optimal solution for each window:

[0072]

[0073] Among them, T [k+1] represents the variable in the k+1th iteration, ρ1 represents the first penalty coefficient, represents the matrix in the current k-th iteration, P [k] Represents the eigenvector of the sample covariance matrix after the kth iteration, U [k] represents the first intermediate variable used to satisfy the constraints, Indicates the calculation of the square of the F norm of the matrix, C [k+1] represents the sample covariance matrix in the k+1th iteration, ρ2 represents the second penalty coefficient, represents the measurement matrix after normalization in the kth iteration, V [k] represents the second intermediate variable used to satisfy the constraint in the kth iteration, represents the measurement matrix after normalization in the k+1th iteration, ρ3 represents the third penalty coefficient, W [k] represents the third intermediate variable used to satisfy the constraints, represents the matrix in the k+1th iteration, P [k+1] represents the eigenvector of the sample covariance matrix calculated by the measurement matrix after the k+1th iteration, U [k+1] Represents the first intermediate variable updated after the k+1th iteration, V [k +1] represents the second intermediate variable in the k+1th iteration, W [k+1] represents the third intermediate variable after the k+1th iteration.

[0074] In this paper, by converting the minimization objective function into a standard ADMM form and performing iterative updates, we can effectively improve optimization efficiency and solution accuracy, handle complex optimization problems, and provide good convergence and stability. This process helps find the optimal solution to the problem of filling missing values, ensuring data accuracy and analytical reliability. It can also handle large-scale data and complex constraints, improving the quality of overall data processing and analysis.

[0075] In a possible implementation, S3052 specifically includes sub-steps S3052A to S3052I:

[0076] S3052A: Initialize the first penalty coefficient ρ1, the second penalty coefficient ρ2, the third penalty coefficient ρ3, the first intermediate variable U, the second intermediate variable V, the third intermediate variable W, and the convergence control limit ε0.

[0077] S3052B: Based on the padded training data, obtain the first variable, the sample covariance matrix, the normalized measurement matrix of the current window, the eigenvector of the sample covariance matrix, and the iterative initial value of the intermediate variable related to the normalized measurement matrix.

[0078] S3052C: Update the first variable:

[0079]

[0080] Where γ represents the first iteration intermediate variable used to complete the calculation in the kth iteration, γ i represents the i-th value of γ, b i represents the i-th column of matrix B, Represents the training data The mean of Represents the training data The sample standard deviation of represents the mean of the i-th column of the first variable, B [k] Represents the second iteration intermediate variable used to complete the calculation in the kth iteration.

[0081] It should be noted that γ is an m-dimensional column vector. To ensure that the w rows of U are all subtracted from the transpose of γ, it is multiplied by a w-dimensional column vector.

[0082] S3052D: Update sample covariance matrix:

[0083] C [k+1] =Q [k] Λ * Q [k]T←

[0084]

[0085] Among them, Q [k] express The corresponding eigenvector, Represents the value of the current window measurement matrix after normalization at the kth iteration, Λ * (C) represents the diagonal matrix of the eigenvalues ​​of the sample covariance matrix in the k+1th iteration, Represents Λ * (C) The value of the diagonal, λ i Represents the operation of calculating the eigenvalue, and obtains the i-th eigenvalue, represents the mean of the eigenvalues ​​calculated from the training data, Represents the sample standard deviation of the eigenvalues ​​calculated from the training data.

[0086] S3052E: Order Initialize the convergence control limit ε X , iteratively update the original measurement matrix until the first preset update condition is reached and the update stops:

[0087]

[0088] in, represents the decision variable at step t in iteration k. , Y t and Y t+1 represents the intermediate variable, C [k+1] represents the sample covariance matrix at the k+1th iteration, V [k+1] represents the variable in the k+1th iteration, I represents the unit matrix, ω represents the weight coefficient, represents the first preset update condition, represents the decision variable at step t+1 in iteration k.

[0089] S3052F: When When , the standardized measurement matrix is ​​updated, that is, the missing position corresponding to the original measurement matrix is ​​updated.

[0090]

[0091] TP=T [k+1] P [k]T ,UP=U [k] P [k]T

[0092] in, Represents an intermediate variable The value of row i and column j, TP ij Indicates the value of TP in row i and column j, UP ij Represents the value of UP row i and column j, Represents the value of the i-th row and j-th column of the measurement matrix after the k+1-th iteration of the current window, Represents the value of W in row i and column j after the kth iteration.

[0093] S3052G: Update the eigenvectors of the sample covariance matrix:

[0094]

[0095] F [k] =T [k+1] +U [k]

[0096] Among them, P [k+1]1 represents the eigenvector of the sample covariance matrix at the k+1th iteration, F [k] Represents the third iteration intermediate variable used to complete the calculation in the k-th iteration.

[0097] S3052H: Update the first intermediate variable, the second intermediate variable, and the third intermediate variable.

[0098] S3052I: Repeat S3052C to S3052H until the second preset update condition is met and the update is stopped to determine the optimal solution in each window:

[0099] O [k+1] -O [k] <ε0

[0100] O [k] =f1(T [k] )+f2(C [k] )

[0101] Among them, O [k+1] -O [k] <ε0 represents the second preset update condition, O [k+1] represents the value of the objective function after the k+1th iteration, O [k] represents the value of the objective function after the kth iteration, T [k] represents the variable after the kth iteration.

[0102] In this paper, the ADMM method excels in processing large-scale data. Through iterative updates, it can efficiently solve large-scale optimization problems and is suitable for missing value filling tasks for complex data. By performing standard ADMM-style iterative updates on the minimized objective function, the missing value filling problem can be solved efficiently and accurately. The initialization step ensures the stability and convergence of the algorithm, while the iterative update process improves computational accuracy and result stability, while also being able to handle complex optimization problems and adapt to data changes. This method not only improves the accuracy of the filled data but also makes the entire optimization process more efficient and flexible.

[0103] S306: averaging the optimal solutions in each window to obtain the optimal solution for the padded training data under the current initialization conditions.

[0104] S307: Repeat S301 to S306 until the third preset update condition is met and the update is stopped, and the optimal solution of the measurement matrix is ​​obtained as the optimal matrix:

[0105] ‖X l+1 -X l ||2<ε1

[0106] Among them, ε1 represents the third preset update condition, X l+1 represents the measurement matrix after the l+1th iteration, X l represents the measurement matrix after the lth iteration, and ||||2 represents the calculation of the matrix's bi-norm.

[0107] It should be noted that the optimal solution of the training data after the padding process is the optimal matrix.

[0108] In this invention, by averaging the optimal solutions for each window, we can reduce noise caused by local data fluctuations, making the resulting optimal solution smoother and more stable. This helps improve the reliability of the filling results. By using a structure similar to the EM algorithm, alternating between calculating statistics, minimizing detection metrics, and updating the measurement matrix, the training set becomes more stable during updates.

[0109] S4: Calculate the statistics of the variables based on the optimal solution of the training data after filling.

[0110] In a possible implementation, S4 specifically includes sub-steps S401 to S405:

[0111] S401: Construct a second single-step sliding window based on the optimal solution of the training data after padding:

[0112]

[0113] Among them, X' k represents the optimal matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, X' represents the optimal matrix, x i ' ,j It represents the optimal solution of the jth measurement variable in the i-th sample in the optimal matrix, that is, the training data after filling processing, 1≤i≤n,1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables.

[0114] S402: Standardize the data in the second single-step sliding window:

[0115]

[0116] in, represents the optimal matrix after normalization at time k, μ0 represents the sample mean, and Σ0 represents the sample standard deviation.

[0117] It should be noted that X' k is a matrix with w rows and m columns, μ0 is an m*1 column vector, that is, the sample mean of each column in the matrix, and 1 is a w*1 matrix with all 1s to implement the mean subtraction operation on each row of the matrix.

[0118] S403: Calculate the covariance matrix of the optimal matrix after normalization:

[0119]

[0120] Among them, cov represents the covariance matrix, C' k represents the sample covariance matrix of the optimal matrix at time k after normalization, T represents the matrix transpose operation, Λ' k Represents the second diagonal matrix, where the element value in the second diagonal matrix is ​​C' k The eigenvalues ​​of , where the eigenvalues ​​are arranged in descending order, P k 'Represents C' k The eigenvectors corresponding to each eigenvalue of .

[0121] S404: Transform the optimal matrix after normalization:

[0122]

[0123] Among them, T' k Represents the second variable at time k, that is, the linear combination of the standardized measurement vectors in the second single-step sliding window.

[0124] S405: In the second single-step sliding window, calculate the statistical information of the second variable to obtain the statistics of the second variable:

[0125] Θ T =[μ T |V T |Γ T ]

[0126]

[0127] Where μ represents the sample mean including the second variable The first-order statistic of , V represents the sample variance including the second variable The second-order statistic of Γ represents the sample skewness including the second variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the second variable, j represents an intermediate variable that traverses each row of the i-th variable, represents the standard deviation of the i-th column of the second variable, represents the sample mean of the i-th variable in the second variable under normal operating conditions.

[0128] In the present invention, by calculating the statistics of the variables (such as mean, variance, skewness, and kurtosis), the quality of the data after filling processing can be evaluated to ensure the accuracy and reliability of the filling results. This helps to determine whether the filled data conforms to the statistical characteristics of the actual data. Statistics (mean, variance, skewness, kurtosis, etc.) can reveal the distribution characteristics and trends of the data. This helps to understand the basic characteristics of the data, such as the central tendency, degree of dispersion, and distribution form of the data, thereby providing a basis for subsequent analysis.

[0129] S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level.

[0130] In a possible implementation, S5 specifically includes sub-steps S501 and S502:

[0131] S501: Calculate the detection index based on the statistics of the second variable:

[0132]

[0133] Among them, D k represents the detection index at time k, represents the statistics of the variables in the k-th sliding window, represents the reference mean of Θ obtained from historical data under normal operating conditions, s represents the number of types of selected statistics, || || p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

[0134] S502: Use the empirical method to determine the control limits using historical data and confidence levels under normal operating conditions.

[0135] It should be noted that the confidence level is also called the significance level.

[0136] For example, if the significance level is chosen to be 0.01, the corresponding control limits can be obtained by making it exceed the detection index of 99% of the training data.

[0137] In the present invention, determining control limits can help identify potential outliers or abnormal patterns in training data. By setting reasonable control limits, it is possible to effectively distinguish between normal data and abnormal data, thereby detecting problems early in industrial processes. Determining control limits can help identify potential outliers or abnormal patterns in training data. By setting reasonable control limits, it is possible to effectively distinguish between normal data and abnormal data, thereby detecting problems early in industrial processes. Using empirical methods and confidence levels to determine control limits, based on statistical principles, ensures that the setting of control limits is scientific and reasonable, thereby enhancing the statistical control capabilities of the system.

[0138] S6: Obtain real-time data with missing values.

[0139] Furthermore, the acquired real-time data containing missing values ​​is integrated into a real-time matrix.

[0140] In the present invention, by acquiring and integrating real-time data, industrial processes can be monitored and analyzed in real time, abnormal situations can be discovered and responded to in a timely manner, and the stability and safety of the process can be ensured.

[0141] S7: Fill in missing values ​​in real-time data.

[0142] Specifically, the real-time matrix is ​​simply filled into a matrix without missing values ​​according to the mean and variance of the training data.

[0143] In this paper, missing value filling ensures the integrity of real-time datasets, ensuring that all data points have values, thereby avoiding data bias caused by missing values ​​during analysis and processing. Filling missing values ​​can reduce gaps or inconsistencies in the dataset, thereby improving data quality and making the data more accurate and representative.

[0144] S8: Determine the optimal solution for the complete data after filling by minimizing the detection index.

[0145] In a possible implementation, S8 specifically includes sub-steps S801 to S805:

[0146] S801: Standardize the complete data after filling:

[0147]

[0148] in, represents the real-time matrix after standardization, μ0 represents the sample mean, Σ0 represents the sample standard deviation, X” represents the real-time matrix, x i ” ,j It represents the jth measurement variable in the i-th sample in the real-time matrix, that is, the complete data after filling processing, 1≤i≤n,1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables.

[0149] It should be noted that X' is a matrix with w rows and m columns, μ0 is an m*1 column vector, i.e., the sample mean of each column in the matrix, and 1 is a w*1 matrix with all 1s to implement the mean subtraction operation on each row of the matrix.

[0150] S802: Calculate the covariance matrix of the normalized real-time matrix:

[0151]

[0152] Where cov represents the covariance matrix, C' represents the sample covariance matrix of the normalized real-time matrix, T represents the matrix transpose operation, Λ' represents the second diagonal matrix, the element values ​​in the second diagonal matrix are the eigenvalues ​​of C' and are arranged in descending order, and P' represents the eigenvector corresponding to the eigenvalue of C'.

[0153] S803: Transform the normalized real-time matrix:

[0154]

[0155] Wherein, T" represents the third variable.

[0156] S804: Optimize the missing values ​​in the real-time matrix by minimizing the objective function constructed with the goal of minimizing the detection index, and determine the optimal solution under each window:

[0157]

[0158] Among them, min means taking the minimum value, represents the real-time matrix after normalization, represents λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the sample covariance matrix calculated from the training data, and λ represents the operation of finding the eigenvalue. represents λ obtained from the training data * The mean of represents μ obtained from the training data * The sample standard deviation of represents μ obtained from the training data * The mean value, μ * represents the mean of the variables calculated from the training data.

[0159] S805: averaging the optimal solutions in each window to obtain the optimal solution for the complete data after filling.

[0160] By minimizing the detection metrics, the present invention can further optimize the filling results when processing the padded data, making the data more accurate and reducing errors introduced by the filling. Standardization and covariance matrix calculation ensure that the data is processed and analyzed under a unified standard, ensuring data consistency and avoiding data inconsistencies caused by different processing methods. The optimal solution in each window is averaged to ensure consistency and comparability of all processed data, reducing data bias.

[0161] S9: Calculate the statistics of the variables based on the optimal solution of the complete data after filling processing.

[0162] In a possible implementation, S9 specifically includes:

[0163] The optimal solution of the complete data after filling is input into the RTCSA model, and the statistical information of the third variable is calculated to obtain the statistics of the third variable:

[0164] Θ' T =[μ' T |V' T |Γ' T ]

[0165]

[0166] Where μ' represents the sample mean including the third variable The first-order statistic of , V' represents the sample variance including the third variable The second-order statistic of Γ' represents the sample skewness including the third variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the third variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the i-th column of the third variable, It represents the sample mean of the i-th variable in the third variable under normal working conditions, which is calculated from the training data.

[0167] The Recursive Transformed Component Statistical Analysis (RTCSA) model is a multivariate statistical process monitoring method for early fault detection. This model constructs a single-step sliding window to obtain orthogonal variables. The statistical information from these variables can reflect important process characteristics, thereby enabling fault detection. The RTCSA model calculates data statistics in real time and compares these statistics with data under normal operating conditions, thereby promptly identifying potential problems or deviations in the system. It is widely used in industrial process monitoring, data analysis, and other fields to ensure system stability and reliability.

[0168] In this paper, the RTCSA (Recursive Variable Statistical Analysis) model allows for comprehensive statistical analysis of the padded data, taking into account statistics such as sample mean, variance, skewness, and kurtosis, providing a more accurate and comprehensive description of the data. Using the optimal solution of the padded complete data as input ensures that the statistical calculations are based on the most reliable data, thereby improving the reliability of the statistical analysis.

[0169] S10: Calculate detection indicators based on the statistics of the variables of the real-time data.

[0170] In a possible implementation, S10 specifically includes:

[0171] According to the statistics of the third variable, the detection index is calculated:

[0172]

[0173] Where D represents the detection index, Θ represents the statistic of the variable in the sliding window, Θ0 represents the reference mean of Θ obtained from historical data under normal working conditions, s represents the number of types of selected statistics, |||| p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

[0174] In this invention, detection indicators are calculated based on the statistics of real-time data variables, providing important functions such as real-time monitoring, anomaly detection, data quality control, and decision support. This helps improve system reliability, responsiveness, and intelligence, while also supporting system optimization and early warning mechanisms. This approach ensures accurate analysis and management of real-time data, providing users with more efficient data processing and decision support.

[0175] S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

[0176] Specifically, determine whether the detection index is greater than the control limit. If D>δ k , δk Indicates the control limit, that is, when the detection index is greater than the control limit, it is judged as an abnormal operating condition, otherwise it is judged as a normal operating condition.

[0177] In this invention, by setting control limits, we can establish clear thresholds to distinguish normal from abnormal conditions, making the anomaly detection process more structured and systematic. By comparing detection indicators with control limits, we can effectively guide maintenance work, prioritize abnormal conditions that exceed control limits, and thus optimize the allocation of maintenance resources.

[0178] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0179] In the present invention, the optimal solution of the padded training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated based on the optimal solution of the padded training data, and the control limits are determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the padded complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal operating condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set.

[0180] Reference Manual Figure 2 , shows a structural schematic diagram of an industrial process fault detection system for missing data provided by the present invention.

[0181] The present invention further provides an industrial process fault detection system 30 for missing data, comprising: a memory 303 and one or more processors 301 .

[0182] One or more application programs are stored in the memory 303 , and the one or more application programs are suitable for being executed by the one or more processors 301 to implement the industrial process fault detection method for missing data described in the method embodiment.

[0183] The industrial process fault detection system 30 for missing data includes a processor 301 and a memory 303 . The processor 301 and the memory 303 are connected, for example, via a bus 302 .

[0184] The structure of the industrial process fault detection system 30 for missing data does not constitute a limitation to the embodiment of the present invention.

[0185] Processor 301 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0186] Bus 302 may include a path for transmitting information between the aforementioned components. Bus 302 may be a PCI bus or an EISA bus, for example. Bus 302 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figure shows only one thick line, but this does not indicate that there is only one bus or only one type of bus.

[0187] The memory 303 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0188] It should be noted that the industrial process fault detection system 30 for missing data can implement the above-mentioned industrial process fault detection method for missing data and can achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate on them.

[0189] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0190] In the present invention, the optimal solution of the padded training data and the complete data is determined by minimizing the detection index, which effectively reduces the false alarm rate when processing missing data and fully restores the integrity of the data. The statistics of the variables are calculated based on the optimal solution of the padded training data, and the control limits are determined in combination with the confidence level. The standard for fault detection can be accurately set. The statistics of the variables are calculated using the optimal solution of the padded complete data, and the detection index is calculated based on the variable statistics of the real-time data. When the detection index exceeds the control limit, it is determined to be an abnormal operating condition. It can not only effectively detect early faults, but also deal with the problem of missing data in the training set and the test set.

[0191] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which can be loaded and executed by a processor to implement the industrial process fault detection method for missing data described in the first aspect.

[0192] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0193] There are a few points to note:

[0194] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0195] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or intervening elements may be present.

[0196] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0197] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for industrial process fault detection with missing data, characterized in that: include: S1: Obtain data with missing values ​​under normal working conditions as training data; S2: Filling missing values ​​in the training data; S3: Determine the optimal solution for the padded training data by minimizing the detection metric; S4: Calculate the statistics of the variables based on the optimal solution of the training data after filling; S5: Determine the control limits based on the statistics of the variables of the training data and the confidence level; S6: Obtain real-time data with missing values; S7: Filling missing values ​​in the real-time data; S8: Determine the optimal solution for the complete data after filling by minimizing the detection index; S9: Calculate the statistics of the variables based on the optimal solution of the complete data after filling; S10: Calculate detection indicators based on the statistics of the variables of the real-time data; S11: When the detection index is greater than the control limit, it is determined to be an abnormal operating condition.

2. The industrial process fault detection method for missing data according to claim 1, characterized in that: The S3 specifically includes: S301: Construct the first single-step sliding window based on the training data after padding: Among them, X k Represents the measurement matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, represents the original measurement matrix, x i,j represents the jth measurement variable in the i-th sample in the original measurement matrix, i.e., the training data after filling, 1≤i≤n, 1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables; S302: Performing normalization processing on the data in the first single-step sliding window: in, represents the normalized measurement matrix at time k, represents the sample mean, represents the sample standard deviation, X k is a matrix with w rows and m columns, μ0 is an m*1 column vector, i.e., the sample mean of each column in the matrix, and 1 is a w*1 matrix with all 1s to implement the mean subtraction operation on each row of the matrix; S303: Calculate the covariance matrix of the normalized measurement matrix: Among them, cov represents the covariance matrix, C k represents the sample covariance matrix of the standardized measurement matrix at time k, T represents the matrix transpose operation, Represents the first diagonal matrix, where the element value in the first diagonal matrix is ​​C k , wherein the eigenvalues ​​are arranged in descending order, Indicates C k The eigenvectors corresponding to each eigenvalue of ; S304: Transform the normalized measurement matrix: in, represents the first variable at time k, i.e., the linear combination of the normalized measurement vectors within the first single-step sliding window; S305: Optimizing the missing values ​​in the original measurement matrix by using a minimization objective function constructed with the goal of minimizing the detection index, and determining the optimal solution for each window in the first single-step sliding window: Among them, min means taking the minimum value, represents the normalized measurement matrix at time k, represents λ obtained from the training data * The sample standard deviation, λ * Represents the eigenvalue of the covariance matrix of the training data sample, λ represents the operation of finding the eigenvalue of the covariance matrix of the sample of the current window, represents λ obtained from the training data * The mean of represents μ obtained from the training data * The sample standard deviation of represents μ obtained from the training data * The mean value, μ * represents the mean of all first variables calculated from the training data; S306: averaging the optimal solutions in each window to obtain the optimal solution for the padded training data under the current initialization conditions; S307: Repeat S301 to S306 until the third preset update condition is met and the update is stopped, and the optimal solution of the measurement matrix is ​​obtained as the optimal matrix: ||X l+1 -X l ||2<ε1 Among them, ε1 represents the third preset update condition, X l+1 represents the measurement matrix after the l+1th iteration, X l represents the measurement matrix after the lth iteration, and ||||2 represents the calculation of the matrix's bi-norm.

3. The industrial process fault detection method for missing data according to claim 2, characterized in that: The S305 specifically includes: S3051: Convert the minimization objective function into a standard ADMM form to obtain a standardized function: in, represents the normalized measurement matrix at time k, and f1 represents the value of T k Related functions, T k represents the variable of the kth window, f2 represents the variable of C k Related functions, C k represents the sample covariance matrix of the kth window, g1 represents the indicator function, when x ij When missing, otherwise x ij represents the value of the i-th row and j-th column of the k-th window, represents the indicator function for the location of missing values, represents the intermediate variable related to the measurement matrix after the normalization of the kth window, g2(P k ) represents the value of P k The indicator function, P k The eigenvector representing the sample covariance matrix of the k-th window; S3052: Iteratively update the normalization function to determine the optimal solution in each window: Among them, T [k+1] represents the variable in the k+1th iteration, ρ1 represents the first penalty coefficient, represents the matrix in the current k-th iteration, P [k] Represents the eigenvector of the sample covariance matrix after the kth iteration, U [k] represents the first intermediate variable used to satisfy the constraints, Indicates the calculation of the square of the F norm of the matrix, C [k+1] represents the sample covariance matrix in the k+1th iteration, ρ2 represents the second penalty coefficient, represents the measurement matrix after normalization in the kth iteration, V [k] represents the second intermediate variable used to satisfy the constraint in the kth iteration, represents the measurement matrix after normalization in the k+1th iteration, ρ3 represents the third penalty coefficient, W [k] represents the third intermediate variable used to satisfy the constraints, represents the matrix in the k+1th iteration, P [k+1] represents the eigenvector of the sample covariance matrix calculated by the measurement matrix after the k+1th iteration, U [k+1] Represents the first intermediate variable updated after the k+1th iteration, V [k+1] represents the second intermediate variable in the k+1th iteration, W [k+1] represents the third intermediate variable after the k+1th iteration.

4. The industrial process fault detection method for missing data according to claim 3, characterized in that: The S3052 specifically includes: S3052A: Initialize the first penalty coefficient ρ1, the second penalty coefficient ρ2, the third penalty coefficient ρ3, the first intermediate variable U, the second intermediate variable V, the third intermediate variable W, and the convergence control limit ε0; S3052B: Obtaining, based on the padded training data, the first variable, the sample covariance matrix, the normalized measurement matrix of the current window, the eigenvectors of the sample covariance matrix, and iterative initial values ​​of intermediate variables associated with the normalized measurement matrix; S3052C: Update the first variable: Where γ represents the first iteration intermediate variable used to complete the calculation in the kth iteration, γ i represents the i-th value of γ, b i represents the i-th column of matrix B, Represents the training data The mean of Represents the training data The sample standard deviation of represents the mean of the i-th column of the first variable, B [k] represents the second iteration intermediate variable used to complete the calculation in the kth iteration; S3052D: Update the sample covariance matrix: C [k+1] =Q [k] L * Q [k]T← Among them, Q [k] express The corresponding eigenvector, Represents the value of the current window measurement matrix after normalization at the kth iteration, Λ * (C) represents the diagonal matrix of the eigenvalues ​​of the sample covariance matrix in the k+1th iteration, Represents Λ * (C) The value of the diagonal, λ i Represents the operation of calculating the eigenvalue, and obtains the i-th eigenvalue, represents the mean of the eigenvalues ​​calculated from the training data, represents the sample standard deviation of the eigenvalues ​​calculated from the training data; S3052E: Order Initialize the convergence control limit ε X , iteratively update the original measurement matrix until the first preset update condition is reached and then stop updating: in, represents the decision variable at step t in iteration k, Y t and Y t+1 represents the intermediate variable, C [k+1] represents the sample covariance matrix at the k+1th iteration, V [k+1] represents the variable in the k+1th iteration, I represents the unit matrix, ω represents the weight coefficient, represents the first preset update condition, represents the decision variable at step t+1 in iteration k; S3052F: When When , the standardized measurement matrix is ​​updated, that is, the missing position corresponding to the original measurement matrix is ​​updated. TP=T [k+1] P [k]T ,UP=U [k] P [k]T in, Represents an intermediate variable The value of row i and column j, TP ij Indicates the value of TP in row i and column j, UP ij Represents the value of UP row i and column j, Represents the value of the i-th row and j-th column of the measurement matrix after the k+1-th iteration of the current window, represents the value of W in row i and column j after the kth iteration; S3052G: Update the eigenvectors of the sample covariance matrix: F [k] =T [k+1] +U [k] Among them, P [k+1] represents the eigenvector of the sample covariance matrix at the k+1th iteration, F [k] represents the third iteration intermediate variable used to complete the calculation in the kth iteration; S3052H: Update the first intermediate variable, the second intermediate variable, and the third intermediate variable; S3052I: Repeat S3052C to S3052H until the second preset update condition is met and the update is stopped to determine the optimal solution in each window: O [k+1] -O [k] <ε0 O [k] =f1(T [k] )+f2(C [k] ) Among them, O [k+1] -O [k] <ε0 represents the second preset update condition, O [k+1] represents the value of the objective function after the k+1th iteration, O [k] represents the value of the objective function after the kth iteration, T [k] represents the variable after the kth iteration.

5. The industrial process fault detection method for missing data according to claim 2, characterized in that: The S4 specifically includes: S401: Construct a second single-step sliding window based on the optimal solution of the training data after padding: Among them, X' k represents the optimal matrix at time k, k represents the sampling time of the latest data in the sliding window, w represents the window width, X' represents the optimal matrix, x' i,j represents the optimal solution of the jth measurement variable in the i-th sample in the optimal matrix, that is, the training data after filling processing, 1≤i≤n,1≤j≤m, n represents the total number of samples, and m represents the total number of measurement variables; S402: Performing normalization processing on the data in the second single-step sliding window: in, represents the optimal matrix after normalization at time k, μ0 represents the sample mean, and Σ0 represents the sample standard deviation; S403: Calculate the covariance matrix of the optimal matrix after normalization: Among them, cov represents the covariance matrix, C' k represents the sample covariance matrix of the optimal matrix at time k after normalization, T represents the matrix transpose operation, Λ' k Represents the second diagonal matrix, where the element value in the second diagonal matrix is ​​C' k The eigenvalues ​​of , wherein the eigenvalues ​​are arranged in descending order, P k 'Represents C' k The eigenvectors corresponding to each eigenvalue of ; S404: Transform the optimal matrix after normalization: Among them, T k ' represents the second variable at time k, i.e., the linear combination of the normalized measurement vectors within the second single-step sliding window; S405: In the second single-step sliding window, calculate the statistical information of the second variable to obtain the statistical quantity of the second variable: I T =[μ T |V T |C T ] Where μ represents the sample mean including the second variable The first-order statistic of , V represents the sample variance including the second variable The second-order statistic of Γ represents the sample skewness including the second variable and sample kurtosis Higher-order statistics, t i represents the i-th column of the second variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the i-th column of the second variable, represents the sample mean of the i-th variable in the second variable under normal operating conditions.

6. The industrial process fault detection method for missing data according to claim 5, characterized in that: The S5 specifically includes: S501: Calculate the detection index based on the statistics of the second variable: Among them, D k represents the detection index at time k, represents the statistics of the variables in the k-th sliding window, represents the reference mean of Θ obtained from historical data under normal working conditions, s represents the number of types of selected statistics, |||| p represents the vector p-norm, represents the sample standard deviation of the variable statistic; S502: Use the empirical method to determine the control limits using historical data and confidence levels under normal operating conditions.

7. The industrial process fault detection method for missing data according to claim 1, characterized in that: S8 specifically includes: S801: Standardize the complete data after filling: in, represents the real-time matrix after normalization, μ0 represents the sample mean, Σ0 represents the sample standard deviation, X” represents the real-time matrix, x″ i,j It represents the jth measurement variable in the i-th sample in the real-time matrix, i.e., the complete data after filling processing, 1≤i≤n, 1≤j≤m, where n represents the total number of samples and m represents the total number of measurement variables; S802: Calculate the covariance matrix of the normalized real-time matrix: Among them, cov represents the covariance matrix, C ” represents the sample covariance matrix of the normalized real-time matrix, T represents the matrix transpose operation, Λ' represents the second diagonal matrix, the elements in the second diagonal matrix are the eigenvalues ​​of C' and are arranged in descending order, and P' represents the eigenvector corresponding to the eigenvalue of C'; S803: Transform the normalized real-time matrix: Wherein, T” represents the third variable; S804: Optimizing the missing values ​​in the real-time matrix by using a minimization objective function constructed with the goal of minimizing the detection index to determine the optimal solution in each window: Among them, min means taking the minimum value, represents the real-time matrix after normalization, represents λ obtained from the training data * The sample standard deviation, λ * represents the eigenvalue of the sample covariance matrix calculated from the training data, and λ represents the operation of finding the eigenvalue. represents λ obtained from the training data * The mean of represents μ obtained from the training data * The sample standard deviation of represents μ obtained from the training data * The mean value, μ * represents the mean of the variables calculated from the training data; S805: averaging the optimal solutions in each window to obtain the optimal solution for the complete data after filling.

8. The industrial process fault detection method for missing data according to claim 7, characterized in that: The S9 is specifically: The optimal solution of the complete data after filling is input into the RTCSA model, and the statistical information of the third variable is calculated to obtain the statistics of the third variable: I 'T =[μ 'T |V 'T |C 'T ] Where μ′ represents the sample mean including the third variable The first-order statistic of , V' represents the sample variance including the third variable The second-order statistic of Γ' represents the sample skewness including the third variable and sample kurtosis Higher-order statistics of t′ i represents the i-th column of the third variable, j represents the intermediate variable used to traverse each row of the i-th variable, represents the standard deviation of the i-th column of the third variable, It represents the sample mean of the i-th variable in the third variable under normal working conditions, which is calculated from the training data.

9. The industrial process fault detection method for missing data according to claim 8, characterized in that: The S10 is specifically: According to the statistics of the third variable, the detection index is calculated: Where D represents the detection index, Θ represents the statistic of the variable in the sliding window, Θ0 represents the reference mean of Θ obtained from historical data under normal working conditions, s represents the number of types of selected statistics, |||| p represents the vector p-norm, represents the sample standard deviation of the variable statistic.

10. An industrial process fault detection system for missing data, characterized in that include: memory and one or more processors; One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the industrial process fault detection method for missing data according to any one of claims 1 to 9.

Images