A KL divergence minor fault detection method based on improved self-attention mechanism

Abstract

The present invention relates to the technical field of industrial process fault detection, and discloses a KL divergence minor fault detection method based on an improved self-attention mechanism, including offline training: principal component analysis modeling based on training data to obtain a load matrix, and calculating a score matrix of training data and a score matrix of test data; performing a KL divergence test based on a sliding window strategy and a score matrix to obtain a KL divergence statistic; introducing an improved self-attention mechanism and a local outlier factor weighting method, and performing a second KL divergence calculation based on the KL divergence statistic to obtain a training weighted score matrix, obtain an attention statistic, and then obtain the final statistics of historical production data; using a kernel density estimation method to estimate control limits; calculating online statistics, and comparing them with control limits to determine whether a fault occurs in the industrial process at the current moment. The present invention is more sensitive to minor fault perception, and has a higher fault detection rate and better robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial process fault detection, and in particular to a KL divergence minor fault detection method based on an improved self-attention mechanism. Background Art

[0002] In modern industrial production, minor faults have low amplitudes, are easily disturbed or drowned by noise, and have extremely weak fault characteristics. With the continuous improvement of industrial automation, the production environment is generally filled with numerous sensors to continuously monitor and control various complex industrial processes. Nevertheless, the increase in the number of sensors is also accompanied by a certain increase in the probability of failure. Ignoring minor deviations in sensors may trigger a series of chain reactions, thereby affecting the performance of the entire production process and even the quality of the final product. Therefore, the detection and diagnosis of minor faults of sensor types has become an indispensable strategy for preventing major accidents and their spread. By perceiving minor faults, effectively planning predictive maintenance plans for industrial equipment and avoiding more serious failures are crucial to ensuring safe, stable and efficient production.

[0003] At present, traditional fault detection methods mainly target significant faults, i.e. faults with large fluctuations in fault parameters and amplitudes, but often have poor detection effects on minor faults. Fault detection technology based on specific models has been widely studied in the research of detecting minor fault perception, with remarkable results. However, modeling methods are highly dependent on accurate modeling of system mechanisms, but it is extremely difficult to build a highly accurate system mechanism model.

[0004] To this end, a large number of studies have been devoted to developing data-driven methods for minor faults to enhance process fault detection and diagnosis. Data-driven methods do not rely on prior knowledge such as mathematical and physical relationships of the system, and a large number of sensors generate a large amount of process data during the detection process, which provides a necessary foundation for research, modeling and monitoring based on data methods. Therefore, in the field of fault detection and diagnosis, data-driven methods have received widespread attention in recent years. Among them, principal component analysis (PCA) in multivariate statistical process monitoring methods is suitable for fault detection in many fields, such as temperature sensing, power systems and wireless sensor networks (WSNs). It can reduce the dimension of complex data while retaining the most important features of the original data. However, this method cannot handle nonlinear problems in actual complex industrial system processes, and principal component analysis relies too much on the changes in the mean and variance of fault data for process detection, but the amplitude of minor faults is low and the variance changes are very weak, which makes it difficult for traditional fault detection methods represented by principal component analysis and canonical variate analysis (CVA) to detect minor faults.

[0005] Therefore, many extended algorithms based on principal component analysis have emerged and applied to actual industrial processes. Divergence can quantify the probability density distribution of normal industrial operation processes and faulty industrial operation processes, such as Kullback-Leibler divergence (KL divergence, KLD) and Jensen-Shannon divergence (JS divergence, JSD). Some studies have combined it with principal component analysis to detect minor faults. This method fully mines the implicit probability distribution information in industrial process data to improve the detection and diagnosis efficiency of minor faults. Experimental results show that the divergence of minor fault perception is more sensitive, which can improve the fault detection rate to a certain extent, and the detection performance is significantly improved compared with the principal component analysis detection method. In addition, the PCA-KLD and PCA-JSD detection methods treat all statistical components equally, and fail to emphasize those KL divergence and JS divergence components that contain more critical fault information. However, the degree of minor faults is not the same at all times. This equal treatment may weaken or mask more valuable information related to the fault, and due to the correlation between data, this may reduce the accuracy of fault detection.

[0006] Therefore, considering that the PCA-KLD fault detection method under the principal component analysis framework may have a tendency of explosive growth of data gradients, this will lead to local detection confusion and affect the accuracy of KL divergence detection; at the same time, this method treats all KL divergence statistical components equally, fails to emphasize important data groups with greater correlation with faults, resulting in a decrease in fault detection rate, low robustness, and failure to detect minor faults in a timely manner. This equal treatment makes fault detection more difficult, and how to assign reasonable weights to enhance fault detection capabilities is one of the problems that need to be solved urgently. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a KL divergence minor fault detection method based on an improved self-attention mechanism.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A KL divergence minor fault detection method based on an improved self-attention mechanism, comprising:

[0010] Offline training: Collect historical production data at each moment of normal industrial operation and divide the historical production data into training data sets And the test dataset ;based on Perform principal component analysis modeling to obtain the load matrix , and calculate the score matrix of the training data in the principal component subspace and the score matrix of the test data ; Based on sliding window strategy and , Perform KL divergence test to obtain historical production data Divergence statistics; introduce improved self-attention mechanism and local outlier factor weighting method, based on Divergence statistics get the training weighted score matrix ; Perform a second KL divergence calculation to obtain the attention statistics , combined with Get the final statistics of historical production data ; For a given impact factor , using the kernel density estimation method to estimate Control limits ;

[0011] Online detection: Collect online production data of the current industrial operation process and use Calculate the score matrix for online production data , calculate online statistics , by comparison and , determine whether a fault occurs in the industrial operation process at the current moment.

[0012] Furthermore, the offline training specifically includes the following steps:

[0013] S11, in normal industrial processes, the collected historical production data includes m observation variables, each observation variable has n moments; the historical production data is divided into training data sets And the test dataset , and standardize the training and test datasets; for The nth training data in for The nth test data in;

[0014] S12, based on the training data set Construct the principal component analysis model and obtain the load matrix , using the eigenvalue cumulative variance contribution rate, determine the number of principal components of the principal component analysis model , through the load matrix Calculate the score matrix of the training data in the principal subspace and the score matrix of the test data ;

[0015] S13, given the width of the sliding window Width, respectively use The fth test score and The fth training score in , calculate the first Width elements of Variance and Variance ,based on and Perform a KL divergence test, slide the sliding window backward, and calculate Divergence Statistics ;

[0016] S14, introduces an improved self-attention mechanism to calculate the query matrix at the i-th moment , weight matrix and value matrix ; Among them, focusing on the elements with test scores in the sliding window obtains ; Use the local outlier factor weighting method to quantify the difference in fault degree and obtain the weight matrix , will be The score matrix of the associated training data As a value matrix ;

[0017] S15, combined , , Get the improved self-attention output matrix in the sliding window at the i-th moment ,Will As the training weighted score matrix ;

[0018] S16, based on the training data set Construct a principal component analysis model and repeat the process from step S12 to step S15 to obtain the fault weighted score matrix , based on the training weighted score matrix and the fault weighted score matrix Perform a second KL divergence calculation and combine it with the obtained attention statistics Get the final statistics of historical production data based on the improved self-attention mechanism KL divergence strategy ;

[0019] S17, for a given impact factor , using the kernel density estimation method to estimate the final statistic Control limits .

[0020] Furthermore, in step S12, the number of principal components of the principal component analysis model is determined by using the eigenvalue cumulative variance contribution rate. , specifically including:

[0021] Starting from the largest eigenvalue in the covariance matrix corresponding to the training data set, the variance contribution rates corresponding to the eigenvalues ​​are accumulated in sequence until the cumulative variance contribution rate reaches the preset threshold :

[0022] ;

[0023] represents the eigenvalue corresponding to the m'th observed variable, Before The eigenvalue corresponding to the jth observed variable in the principal component, Before The cumulative variance contribution rate corresponding to the principal element is , .

[0024] Further, in step S12, the load matrix Calculate the score matrix of the training data in the principal subspace and the score matrix of the test data , specifically including:

[0025] Through the load matrix and training dataset The corresponding matrix , calculate the training data score matrix in the principal component subspace ;

[0026] ;

[0027] Through the load matrix And the test dataset The corresponding matrix , calculate the test data score matrix in the principal component subspace ;

[0028] ;

[0029] for The training scores, for The Test scores.

[0030] Further, in step S16, the combined attention statistics Get the final statistics of historical production data based on the improved self-attention mechanism KL divergence strategy , specifically including:

[0031] The attention statistics Directly assign weights, focus on important fault information, and obtain the final statistics based on the KL divergence strategy of the improved self-attention mechanism :

[0032] ;

[0033] in, is the weight coefficient in the weight matrix The average matrix of the weight matrix corresponding to the entire fault industrial operation process; Represent the training weighted score matrix The training weighted score and fault weighted score matrices in The weighted score of the faults in .

[0034] Furthermore, the online detection specifically includes the following steps:

[0035] S21, continuous collection of online production data , forming an online production dataset , for Middle Online production data, for The total number of online production data in; when satisfied When the online production data is standardized, the load matrix Calculate the score matrix for online production data ; Indicates the sliding window width of online production data;

[0036] S22, by calculating the mean of the online production data in each sliding window and variance , get online production data Divergence Statistics ,when When , online detection starts; where k is the neighbor value in the local outlier factor weighting method;

[0037] S23, calculate the query matrix of online production data in the sliding window at the oth moment , represents the oth moment Divergence statistics, using the local outlier factor weighting method to obtain the weight matrix of online production data , the score matrix of online production data Value matrix as online production data ;

[0038] S24, based on the query matrix corresponding to the online production data at the oth moment in the sliding window , weight matrix and value matrix , get the improved self-attention output matrix of the online production data in the sliding window at the oth moment , and recorded as the online weighted score matrix ;

[0039] S25, according to Perform a second KL divergence calculation: Calculate the online attention statistics using the KL divergence test formula ; The weight matrix of online production data Take the average value to get the online weight matrix , and then get the online statistics ; Represents the training weighted score matrix The training weighted score in , Represents the online weighted score matrix Online weighted score in ;

[0040] S26, judging online statistics Is it greater than the control limit? If yes, it is determined that a fault has occurred in the current industrial operation process; if not, steps S22 to S26 are repeated to implement online fault detection.

[0041] Compared with the prior art, the beneficial technical effects of the present invention are:

[0042] The present invention combines the local outlier factor weighting method and the improved self-attention mechanism to strengthen the fault feature information, use high weights to focus on important fault information, use low weights to ignore normal industrial operation process data, and obtain innovative statistics based on the improved self-attention mechanism KL divergence strategy. , improves the detection rate of minor faults, solves the problem of low detection rate of principal component analysis fault detection method and canonical variable analysis fault detection method, is more sensitive to fault changes, overcomes the problem of local detection confusion caused by equal treatment of KL divergence components by a single divergence, and is successfully applied to the Tennessee Eastman process (TE), with the best detection effect, which can greatly reduce the safety hazards caused by minor faults in industrial processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The present invention is a flowchart of a KL divergence minor fault detection method based on an improved self-attention mechanism in an embodiment of the present invention.

[0044] FIG. 2 (a) to FIG. 2 (e) are effect diagrams of a detection method for a minor fault of the Tennessee Eastman process in an embodiment of the present invention; FIG. 2 (a) is an effect diagram of a fault detection method based on principal component analysis (PCA). Statistic and SPE statistic effect diagram; Figure 2 (b) is the fault detection method based on canonical variable analysis (CVA) Figure 2 (c) is the effect diagram based on KL divergence detection; Figure 2 (d) is the effect diagram based on JS divergence detection; Figure 2 (e) is the effect diagram of the KL divergence minor fault detection method based on the improved self-attention mechanism.

[0045] Figures 3(a) to 3(e) are effect diagrams of a detection method for a minor fault of the Tennessee Eastman process in an embodiment of the present invention. Figure 3(a) is an effect diagram of a fault detection method based on principal component analysis (PCA). Figure 3 (b) shows the effect of the fault detection method based on the canonical variable analysis (CVA). Figure 3 (c) is the effect diagram based on KL divergence detection; Figure 3 (d) is the effect diagram based on JS divergence detection; Figure 3 (e) is the effect diagram of the KL divergence minor fault detection method based on the improved self-attention mechanism.

[0046] Figure 4 This is a fault detection rate comparison chart of different fault detection methods for minor fault three and minor fault fifteen of the Tennessee Eastman process in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to clearly and completely describe the technical solution of the KL divergence minor fault detection method based on the improved self-attention mechanism in the embodiment of the present invention, it will be described in detail below in conjunction with the drawings in the embodiment of the present invention. Obviously, the embodiment described below is a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0048] The present invention provides a KL divergence minor fault detection method based on an improved self-attention mechanism, comprising:

[0049] Step 1: Collect historical production data of normal industrial operation process, including m observation variables, each observation variable is observed and recorded at n different time points, forming a Dimensional production data matrix .

[0050] The principal component analysis algorithm is used to model the historical production data. In order to eliminate the influence of significant variables in the dimensionality reduction process, After normalization, the original production data matrix The mean and standard deviation of each column in and The normalized production data matrix is ​​expressed as :

[0051] ; (1)

[0052] Through linear transformation, the historical production data containing multiple observation variables are reduced to a new low-dimensional space, and the data with the most significant features are effectively extracted to facilitate fault detection and diagnosis.

[0053] Next, Perform eigenvalue decomposition to obtain the loading matrix :

[0054] ; (2)

[0055] in, Include All eigenvalues ​​of , and require , and the unit eigenvector matrix corresponding to the corresponding eigenvalue is recorded as , represents the covariance matrix, represents the eigenvector matrix.

[0056] Therefore, the score matrix for:

[0057] ; (3)

[0058] The cumulative contribution rate of variance is determined based on the eigenvalue of the sample matrix in the principal component analysis theory. The ratio of the eigenvalue corresponding to each principal component to the total eigenvalue is called the contribution rate of the principal component to the total variance of the sample. The number of principal components is determined when the cumulative contribution rate percentage of the principal component exceeds the preset threshold of 90%. , the formula is as follows:

[0059] ; (4)

[0060] Load Matrix , the cumulative variance contribution rate determines the previous Principal Subspace , and the remaining The residual subspace composed of components .

[0061] Established in the principal subspace The statistical test is performed on the statistics, and the fluctuation of the principal vector modulus of the principal subspace is used to reflect the changes of multiple variables and monitor the process. The statistic can be expressed as:

[0062] ; (5)

[0063] In addition, SPE represents the squared prediction error in the residual subspace and can be defined as:

[0064] ; (6)

[0065] Among them, I is The identity matrix of .

[0066] At a given impact factor In the case of Control limits for statistics:

[0067] ; (7)

[0068] in, The degrees of freedom are and On the F distribution Critical point. In industrial processes, the following logical conditions are used to determine whether a failure has occurred:

[0069] ; (8)

[0070] Upper control limit of the SPE statistic It can be calculated by its approximate distribution:

[0071] ; (9)

[0072] Among them, the intermediate variable , intermediate variable . is a normal distribution at the test level If , then the industrial process is a faulty industrial operation process. If , then the industrial process is a normal industrial operation process.

[0073] Step 2: The present invention introduces a fault detection method based on KL divergence. The method originates from the concept of KL divergence in probability theory and information theory, and defines a measurement method for measuring the degree of difference between two probability density functions.

[0074] definition and are two different probability density functions of the continuous variable y. KL divergence can quantify the difference between two probability distributions. It measures the similarity between two distributions. is defined as:

[0075] ; (10)

[0076] because , so KL divergence is asymmetric, so if the data has the same probability distribution but the distance is calculated using the formula of the opposite distribution, KL divergence will produce two different asymmetric quantities.

[0077] Using the obtained load matrix , calculate the score matrix of the training data in the principal element subspace and the score matrix of the test data . and They are the matrices corresponding to the training data set and the test data set respectively.

[0078] Given a sliding window width Width, use and The first Width elements of estimate the kth training score and the kth test score The variance of and .

[0079] and Satisfies Gaussian distribution and ,in and are the mean and standard deviation of the normal industrial operation process, and are the mean and standard deviation of the faulty industrial operation process, respectively.

[0080] and The KL divergence between The is:

[0081] ; (11)

[0082] The present invention sets the sliding window width by comprehensively considering the statistical characteristics of fault data and the dynamic sliding window width in fault detection. .

[0083] According to the sliding window width selection theorem, a large sliding window width can ensure that more minor faults are detected and the detectability of the method is guaranteed. However, a sliding window width that is too large may cause a large time delay. A sliding window width that is too small may cause the method to fail to detect minor faults or misjudge the method itself, thereby failing to achieve the effect of minor fault detection.

[0084] Generally speaking, the magnitude of minor faults is often small, and the positive and negative fluctuation noise can be offset by accumulating samples over a period of time. Therefore, the average of the normal industrial operation process and the faulty industrial operation process is and satisfy: and .

[0085] To further deduce, assume that and The distributions satisfy and . Therefore, the KL divergence estimate can be further simplified as:

[0086] ; (12)

[0087] Then you can get the following results:

[0088] ; (13)

[0089] Defining intermediate variables , through multiple simplifications we get:

[0090] ; (14)

[0091] analyze The relationship between the KL divergence estimation and the sensor accuracy degradation failure is taken as an example. When the variance of the fault data is greater than the variance of the normal data, that is, When , the KL divergence value increases with the increase of the variance of the fault data. Therefore, for some faults with large variance, the KL divergence value is positively correlated with the fault level.

[0092] when , then the variance of the fault data is equal to the variance of the normal data, , so no malfunction occurs.

[0093] when When the difference between the variance of the faulty data and the variance of the normal data increases, the KL divergence value is also a positive development feature related to the degree of deviation. Therefore, the KL divergence is sensitive to the detection of small faults.

[0094] Step 3: The present invention provides an improved self-attention mechanism KL divergence fault detection method, and the self-attention mechanism principle implementation includes:

[0095] The essence of improving the self-attention mechanism is to achieve efficient allocation of key information resources. The core goal of the attention mechanism is to overcome the problem of important information being buried by assigning a learnable weight to each part of the data group sequence.

[0096] There are many forms of attention mechanisms, the basic definition is:

[0097] ; (15)

[0098] Where Q represents the query matrix, K represents the weight matrix, and V represents the value matrix.

[0099] The principle diagram of the improved self-attention mechanism KL divergence fault detection method in the present invention is as follows: Figure 1 As shown:

[0100] In the present invention, the query matrix Q of the self-attention mechanism KL divergence detection represents the KL divergence statistic based on principal component analysis. Since the attention function can be described as mapping the query matrix and a set of key values ​​one by one to the output matrix, the weight of the self-attention mechanism essentially depends on the corresponding correlation between the query matrix Q and K.

[0101] In the present invention, the value matrix is ​​assigned The weight of will be obtained by calculating the local outlier factor connecting the KL divergence and the fault degree. This follows the attention mechanism law, meets the weight allocation requirements in the self-attention mechanism, and makes the weight assignment in fault detection more reasonable.

[0102] Step 4: The present invention provides a method for weighting outliers based on local outlier factors as an attention weight matrix for improving the self-attention mechanism.

[0103] The Local Outlier Factor (LOF) detection algorithm is an unsupervised anomaly detection method. The local outlier factor realizes outlier identification by outlier detection based on data point density. The local outlier factor can calculate the local density deviation of a given data point relative to its neighbors, and considers samples with density much lower than that of its neighbors as outliers. This method has a high fault detection rate and good robustness.

[0104] Outlier detection based on local outlier factor (LOF) is also often used in fault detection of industrial processes. Its application in fault detection can quantify the degree of local deviation of data points and effectively detect those points that are significantly different from most data points in density distribution. These points may indicate minor changes in the system, such as equipment performance degradation, slight component loss, or initial operational abnormalities.

[0105] In the present invention, the local outlier factor focuses on local anomalies, and the KL divergence itself can measure the difference between the probability distribution of normal data and fault data. Therefore, the combination of the local outlier factor and the KL divergence can quantify the difference in fault degree, and give reasonable weights to the statistics to improve the accuracy of fault detection and the ability of early fault warning, more comprehensively analyze the relationship between data points, and improve the reliability and effectiveness of the fault prediction model.

[0106] Self-attention mechanism has not been fully popularized in data-driven fault detection, but its ability to focus on important information is worth paying attention to. The self-attention mechanism model assigns different attention weights to each data point, identifies and emphasizes the features that are most important for the current prediction task, and thus more effectively captures information that may characterize the fault.

[0107] The self-attention mechanism distinguishes fault information from normal information in data points by assigning different weights, identifying the data segments most relevant to potential faults. Compared with traditional machine learning methods, this weight assignment mechanism can not only improve the accuracy of fault detection.

[0108] In the present invention, the self-attention mechanism is combined with the local anomaly factor algorithm and fused with the KL divergence to quantify the differences between data points, thereby capturing minor faults in the industrial monitoring process.

[0109] The key information emphasized by the self-attention mechanism can distinguish the subtle differences between normal and abnormal conditions, and provide early warning before the fault develops to affect production efficiency and product quality. This early warning capability is crucial to ensuring the safety and reliability of industrial systems.

[0110] In the improved self-attention mechanism, the local outlier factor algorithm and KL divergence test are combined to obtain the self-attention mechanism weight matrix , the specific steps include:

[0111] Calculate the KL divergence for each sample in the test dataset The Euclidean distance to other samples ,in yes The fth neighbor of . kth distance is defined as The Euclidean distance to the kth neighbor.

[0112] Calculation Sample and The reachable distances of its k nearest neighbors :

[0113] ; (16)

[0114] Compute the local reachability density for each sample:

[0115] ; (17)

[0116] Finally, calculate the local outlier factor of all samples :

[0117] . (18)

[0118] In the present invention, the local outlier factor algorithm calculates an outlier factor for each point in the data set, and determines whether it is an outlier factor by judging whether the outlier factor is close to 1. If the outlier factor is much greater than 1, it is considered an outlier factor, and if it is close to 1, it is a normal point.

[0119] Since the KL divergence itself can measure the difference between the probability distributions of normal data and fault data, the present invention uses the KL divergence as the data set of the local outlier factor algorithm.

[0120] Divergence Statistics It is used to compare the density of the statistic at a certain time point with the density of its neighbors. If the density at this time point is smaller than that of its neighbors, it is an abnormal time point, that is, in the entire KL divergence statistic, Compared with normal data moments, this moment is an obvious outlier, so the data at this moment is regarded as fault data, and a correspondingly larger weight is given to it in the self-attention mechanism.

[0121] Otherwise, if the density at that moment is greater than the density of its neighbors, that is, the statistic at that moment is Compared with the local density deviation at normal times, This is a normal time point, so this moment is classified as a normal industrial operation process.

[0122] Finally, the local outlier factor weight coefficient that can quantify the difference in fault degree can be obtained.

[0123] In the improved self-attention mechanism of the present invention, combined with formula (18), at the i-th moment, the weighted attention matrix of the samples at the current moment and the next k moments in the sliding window is recorded as , is the weight coefficient obtained by the local outlier factor .

[0124] Because the local reachable density of the KL divergence component is the inverse of the average reachable distance of the nearest neighbor based on the KL divergence, the greater the relative distance at this moment, the smaller the density, and the larger the amplitude of the corresponding local outlier factor weight coefficient; otherwise, the smaller the relative distance at this moment, the greater the density, and the smaller the amplitude of the corresponding local outlier factor weight coefficient.

[0125] Therefore, in the present invention, if the query matrix Q at a certain time point The density of the nearest k neighbors is basically the same as its own density. The time is a normal industrial operation process, then the weight coefficient The closer it is to 1, the smaller the weight given to the fault;

[0126] otherwise, At this moment, the relative distance is far, the density is small, and the judgment point is the fault moment, the corresponding weight coefficient The larger the value, the higher the fault weight is assigned.

[0127] Therefore, in fault detection, the local outlier factor weighting method conforms to the corresponding correlation between the query matrix Q and the weight matrix K in the self-attention mechanism, follows the rule of assigning weights to key faults in the attention mechanism, and makes the weight assignment in fault detection more reasonable.

[0128] Step 5. In the present invention, the value matrix V of the improved self-attention mechanism depends on the score matrix T which is closely related to the KL divergence matrix. This is because the weight coefficient local outlier factor acting on the KL divergence can quantify the difference in the degree of faults, and the KL divergence is obtained through the T variance. Therefore, the score matrix T corresponds to the weight matrix K one by one, and there is a direct mapping relationship between them.

[0129] The value matrix V is recorded as :

[0130] ; (20)

[0131] in The score matrix T from the principal component analysis model.

[0132] Therefore, the settings of Q, K, and V in the present invention are consistent with the idea of ​​the self-attention mechanism.

[0133] The weights are adjusted to strengthen the dependencies of important fault information in the global fault data set. Therefore, the fault attention mechanism has good scalability and robustness.

[0134] So far, the three matrices Q, K, and V corresponding to the samples at each moment have been calculated. Further, the improved self-attention output matrix at each moment is obtained .

[0135] ;(twenty one)

[0136] in, and are the score data and weight coefficient at the i-th moment respectively. Therefore, based on the combination of the local outlier factor weighting method and the improved self-attention mechanism, the training weighted score matrix can be obtained: : In the principal subspace, train the weighted score matrix It is expressed as:

[0137] .(twenty two)

[0138] In step 6, the present invention performs two KL divergence calculations. The first KL divergence calculation is performed in step 2, and the second KL divergence calculation is based on the improved self-attention mechanism.

[0139] From step 5, we get a new set of training weighted score matrices , similarly, based on the training data set Construct the principal component analysis model and repeat the above process to obtain the fault weighted score matrix . Again, we use the training weighted score matrix The mean ,variance and the attention failure score matrix The mean ,variance To calculate the attention statistic According to formula (11) in step 2, we can get:

[0140] ;(twenty three)

[0141] In the present invention, it is equivalent to calculating the KL divergence twice. The first calculation is to combine the KL divergence and the local outlier factor to obtain the weight coefficient for measuring the degree of fault. , so as to be used in the attention mechanism to construct the training weighted score matrix The second time is to find the attention statistic that is more capable of fault detection after being weighted according to the fault degree. .

[0142] Because the final output of the attention mechanism is the training weighted score matrix , the model focuses on the fault data, thereby retaining the key data and serving as The input matrix can achieve high-quality optimization of the input matrix.

[0143] Furthermore, the attention statistics obtained by the second KL divergence calculation are Direct weighting, that is, focusing on important fault information with high weight, can obtain the final statistics based on the KL divergence strategy of the improved self-attention mechanism. :

[0144] ;(twenty four)

[0145] in, is the weight coefficient in the weight matrix The averaged matrix of the weight matrix corresponding to the entire fault industrial operation process.

[0146] because It comes from the local outlier factor. If the local outlier factor at this moment is larger, it is considered to be an outlier factor, and the degree of failure at this moment is greater. If the weight coefficient at this moment is The smaller it is, the more it is considered to be a normal industrial operation process, and the smaller the degree of failure at that moment. An improved self-attention mechanism strategy that gives greater weight to more significant faults is met.

[0147] The KL divergence fault detection flow chart based on the improved self-attention mechanism is as follows Figure 1 shown.

[0148] Step 7: In the present invention, a kernel density estimation algorithm is used to determine the control limits of the statistic. In actual industrial processes, complex data often contain non-Gaussian features, which leads to the inaccuracy of traditional control limit determination methods.

[0149] The kernel density estimation of the present invention can handle distributions of arbitrary shapes, and the kernel density estimation (KDE) can establish reasonable control limits for the attention optimization divergence. Setting the influence factor , using the kernel density estimation (KDE) method to estimate The control limits for the statistics are as follows:

[0150] ; (25)

[0151] in, is the probability density function of the random variable z.

[0152] The present invention applies the kernel density estimation (KDE) limit method to KL divergence, JS divergence and improved attention divergence And other statistics.

[0153] In practical applications, comparison With control limits , the following logic is used for fault detection:

[0154] ; (26)

[0155] Among them, v is the query value of the production data observation variable.

[0156] If the final statistic Below control limit , it indicates that the current industrial operation process is a normal industrial operation process;

[0157] If the final statistic Above control limit , it indicates that the current industrial operation process is a faulty industrial operation process.

[0158] In summary, the present invention first uses the normalized training data of the normal industrial operation process to establish a principal component analysis model, and obtains the load matrix by eigenvalue decomposition , through the load matrix Get the score matrix of training data and test data and , established in the principal subspace Statistics, establish SPE statistics in the residual subspace for fault detection, and In the case of Control limits of statistics , similarly, the upper control limit of the SPE statistic is calculated by approximating the distribution. Combined with the sliding window strategy and using the score matrix, the KL divergence estimate is obtained and used as the query matrix of the improved self-attention mechanism; the weight matrix of the improved self-attention mechanism is obtained by the local outlier factor weighting method. Because the weight of the self-attention mechanism essentially depends on the corresponding correlation between the query matrix Q and the weight matrix K, it can be calculated by connecting the local outlier factor of the relationship between the KL divergence and the degree of fault. This follows the law of the attention mechanism, meets the requirements of weight distribution in the self-attention mechanism, and makes the weight assignment in fault detection more reasonable. After obtaining the query matrix Q, weight matrix K, and value matrix V, the training weighted score matrix output by the improved self-attention mechanism is calculated , the second time, KL estimation is used to calculate the attention statistics with better fault detection ability after fault degree weighting , combined with the high-weight focus on important fault information strategy, the final statistics of the improved self-attention mechanism are obtained . Set the impact factor , using kernel density estimation to establish reasonable control limits for the attention optimization divergence . Compare the statistic to the control limits to determine whether the process is faulty.

[0159] In order to fully describe the KL divergence minor fault detection method based on the improved self-attention mechanism in the embodiment of the present invention, the proposed scheme is applied to the Tennessee Eastman (TE) process for a specific and detailed description.

[0160] The Tennessee Eastman simulation platform is an open chemical model simulation platform based on the actual chemical joint reaction process, which is used to study process control. It consists of multiple operating units such as a continuous stirred reactor, a dephlegmator, a gas-liquid separation tower, a stripping tower, a reboiler and a centrifugal compressor. The Tennessee Eastman process contains 12 control variables and 41 measurement variables. It has a total of 21 preset faults, including 16 known faults and 5 unknown faults. The sampling interval of training samples and test samples is set to 3min. 500 samples when the process is running smoothly are collected as training samples. When sampling the fault data set, the process starts to run in a fault-free state. After running smoothly for 8 simulation hours, a fault is added (that is, the first 160 sample points are fault-free samples). The fault is 21 preset faults, and it is collected until the 500th time point, generating a total of 21 sets of fault data with different faults. The flow diagram of the Tennessee Eastman process is shown in Figure 2.

[0161] The Tennessee Eastman simulation platform has four gaseous reactants A, C, D and E, and an inert component B. The reactants and inert components are fed into the reactor to generate products G and H. The Tennessee Eastman process system satisfies the following formula:

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] The present invention will be applied to two extremely challenging fault conditions in the Tennessee Eastman process, namely, minor fault three and minor fault fifteen, for simulation testing to verify the effectiveness of the method proposed in the present invention.

[0167] In the Tennessee Eastman process, the collected training data is preprocessed and a principal component analysis model is established to obtain the principal component subspace and residual subspace. , and the control line is set using kernel density estimation (KDE). Since minor faults are assigned low values ​​and are susceptible to noise interference, traditional detection methods are difficult to detect in time. Therefore, in order to verify the effectiveness of the method proposed in this invention, challenging minor faults 3 and 15 in the Tennessee Eastman process are selected for simulation verification. The fault descriptions of minor faults 3 and 15 are shown in Table 1.

[0168] Table 1 Fault descriptions of minor faults 3 and 15 in the Tennessee Eastman process

[0169]

[0170] 1. Minor fault three:

[0171] For the minor fault three in the Tennessee Eastman process, a minor step fault is introduced in the feed temperature of the gas reactant D. Because the minor fault three has little effect on the Tennessee Eastman process, the fault data has almost no change in the mean and variance, so the traditional data-driven method cannot detect the minor fault three.

[0172] The KL divergence and JS divergence statistics are obtained using the score matrix, and then the final statistics based on the improved self-attention mechanism KL divergence strategy are calculated according to the above steps. The fault detection effects of the fault detection method based on principal component analysis (PCA), the fault detection method based on canonical variate analysis (CVA), the KL divergence fault detection method based on the PCA framework (PCA-KLD), the JS divergence fault detection method based on the PCA framework (PCA-JSD) and the fault detection method of the present invention are compared. The experimental results show that the fault detection method of the present invention is more sensitive to minor faults in complex industrial systems, has more accurate detection, and has the highest fault detection rate.

[0173] The present invention uses the fault detection rate (FDR) and the fault false alarm rate (FAR) to quantitatively evaluate the performance of all fault detection methods. The specific formula is as follows:

[0174] ;

[0175] ;

[0176] in, represents the number of normal samples that are misdiagnosed as faults, represents the total number of normal samples, represents the number of detected fault samples, Indicates the total number of failed samples.

[0177] From Figure 2 (a) to Figure 2 (e), it can be seen that the fault detection method based on principal component analysis (PCA) Statistics ( ) and SPE statistics ( ) are 7.06% and 12.06% respectively. By introducing divergence in the PCA framework, the statistics of the KL divergence fault detection method based on the PCA framework (PCA-KLD) and the JS divergence fault detection method based on the PCA framework (PCA-JSD) have almost the same effect, which improves the fault detection rate to a certain extent, but the detection effect is still not ideal. In the principal component subspace, the highest fault detection rates are 68.24% and 67.90% respectively. Statistics ( ) and SPE statistics ( ) have a fault detection rate of 3.67% and 0% respectively. However, compared with the previous traditional fault detection and mainstream detection methods, the KL divergence micro-fault detection method based on the improved self-attention mechanism proposed in this invention focuses on more key information of micro-faults, reduces the attention weight on normal data, and enhances the fault feature learning ability of the query matrix.

[0178] Finally, in the principal component subspace, the fault detection rate of the final statistic based on the improved self-attention mechanism KL divergence strategy is as high as 96.18%, far exceeding the KL divergence fault detection method based on the PCA framework (PCA-KLD). Therefore, the final statistic based on the improved self-attention mechanism KL divergence strategy can quantify the weight of fault information, making the fault weight allocation more effective and having higher adaptability and stability.

[0179] 2. Minor fault 15:

[0180] In the present invention, the minor fault 15 of the Tennessee Eastman process is caused by a minor stuck fault of the condenser cooling water valve. Fault 15 is a fault with a relatively small impact on the Tennessee Eastman process, so it is also very challenging to successfully detect Fault 15.

[0181] Figure 3(a) to Figure 3(e) show the fault detection method based on principal component analysis (PCA). Statistics and SPE statistics, fault detection method based on canonical variate analysis (CVA) Statistics and SPE statistics, KL divergence, JS divergence and the final statistics based on the improved self-attention mechanism KL divergence strategy ( ) are compared to demonstrate the effectiveness of the KL divergence micro-fault detection method based on the improved self-attention mechanism in the present invention.

[0182] As shown in Figure 3 (a), the fault detection method based on principal component analysis (PCA) The statistics and SPE statistics can hardly detect minor faults, and the fault detection rates are 1.76% and 8.24% respectively. In the principal component subspace, Figure 3 (b) shows the fault detection method based on canonical variable analysis (CVA) Statistics and SPE statistics, the fault detection rates are 0.51% and 4.49% respectively, and the effect of traditional fault detection methods is not ideal. Figure 3 (c) and Figure 3 (d) respectively show the detection effects of the KL divergence fault detection method based on the PCA framework (PCA-KLD) and the JS divergence fault detection method based on the PCA framework (PCA-JSD). Although the KL divergence and JS divergence statistics can detect minor faults, the effect is not very good, the fault detection rate is not high, and the highest fault detection rate does not exceed 40%.

[0183] Compared with the final statistics based on the improved self-attention mechanism KL divergence strategy in the present invention , the statistics of KL divergence and JS divergence are nearly 50% lower. As shown in Figure 3 (e), From the 201st moment onwards, minor faults can be detected continuously. In the principal component subspace and in the residual subspace, the fault detection rates are as high as 88.82% and 96.76%, respectively.

[0184] Figure 4 are the fault detection rate (FDR) results for two types of small faults in the Tennessee Eastman process case.

[0185] Figure 4 The fault detection rates (FDR) of five fault detection methods are compared. The maximum values ​​of the statistics of the KL divergence fault detection method based on the PCA framework (PCA-KLD) and the JS divergence fault detection method based on the PCA framework (PCA-JSD) in the principal component subspace and the residual subspace are expressed as: , , and In the two minor faults of the Tennessee Eastman process described above, The statistic has the highest fault detection rate, has the best monitoring effect in detecting minor faults, and is more sensitive in sensing faults. The fault detection rates of the two methods are significantly higher than those of the detection methods under the principal component analysis framework, and significantly higher than the maximum values ​​of the KL divergence and JS divergence statistics FDR. and And significantly higher than The FDR of the statistic. Figure 4 middle, and They represent the maximum values ​​of the final statistics based on the improved self-attention mechanism KL divergence strategy in the residual subspace and the principal component subspace respectively.

[0186] Therefore, the improved self-attention mechanism KL divergence fault detection method in the present invention can extract effective fault information and construct a new statistic This statistic can overcome the problem of KL divergence that treats fault information equally, leading to detection confusion. It is more effective in improving the FDR of minor fault detection, is more sensitive to minor fault changes, and has the best effect.

Claims

1. A KL divergence minor fault detection method based on an improved self-attention mechanism, characterized in that: include: Offline training: Collect historical production data at each moment of normal industrial operation and divide the historical production data into training data sets and test dataset ;based on Perform principal component analysis modeling to obtain the load matrix , and calculate the score matrix of the training data in the principal component subspace and the score matrix of the test data ; Based on sliding window strategy and , Perform KL divergence test to obtain historical production data Divergence statistics; introduce improved self-attention mechanism and local outlier factor weighting method, based on Divergence statistics get the training weighted score matrix ; Perform a second KL divergence calculation to obtain the attention statistics , combined with Get the final statistics of historical production data ; For a given impact factor , using the kernel density estimation method to estimate Control limits ; Online detection: Collect online production data of the current industrial operation process and use Calculate the score matrix for online production data , calculate online statistics , by comparison and , determine whether a fault occurs in the industrial operation process at the current moment; specifically include the following steps: S21, continuous collection of online production data , forming an online production dataset , for Middle Online production data, for The total number of online production data in; when satisfied When the online production data is standardized, the load matrix Calculate the score matrix for online production data ; Indicates the sliding window width of online production data; S22, by calculating the mean of the online production data in each sliding window and variance , get online production data Divergence Statistics ,when When , online detection starts; where k is the neighbor value in the local outlier factor weighting method; S23, calculate the query matrix of online production data in the sliding window at the oth moment , represents the oth moment Divergence statistics, using the local outlier factor weighting method to obtain the weight matrix of online production data , the score matrix of online production data Value matrix as online production data ; S24, based on the query matrix corresponding to the online production data at the oth moment in the sliding window , weight matrix and value matrix , get the improved self-attention output matrix of the online production data in the sliding window at the oth moment , and recorded as the online weighted score matrix ; S25, according to Perform a second KL divergence calculation: Calculate the online attention statistics using the KL divergence test formula ; The weight matrix of online production data Take the average value to get the online weight matrix , and then get the online statistics ; Represents the training weighted score matrix The training weighted score in , Represents the online weighted score matrix Online weighted score in ; S26, judging online statistics Is it greater than the control limit? If yes, it is determined that a fault has occurred in the current industrial operation process; if not, steps S22 to S26 are repeated to implement online fault detection.

2. The KL divergence minor fault detection method based on the improved self-attention mechanism according to claim 1 is characterized in that: The offline training specifically includes the following steps: S11, in normal industrial processes, the collected historical production data includes m observation variables, each observation variable has n moments; the historical production data is divided into training data sets and test dataset , and standardize the training and test datasets; for The nth training data in for The nth test data in; S12, based on the training data set Construct the principal component analysis model and obtain the load matrix , using the eigenvalue cumulative variance contribution rate, determine the number of principal components of the principal component analysis model , through the load matrix Calculate the score matrix of the training data in the principal subspace and the score matrix of the test data ; S13, given the width of the sliding window Width, respectively use The fth test score and The fth training score in , calculate the first Width elements of Variance and Variance ,based on and Perform a KL divergence test, slide the sliding window backward, and calculate Divergence Statistics ; S14, introduces an improved self-attention mechanism to calculate the query matrix at the i-th moment , weight matrix and value matrix ; Among them, focusing on the elements with test scores in the sliding window obtains ; Use the local outlier factor weighting method to quantify the difference in fault degree and obtain the weight matrix , will be The score matrix of the associated training data As a value matrix ; S15, combined , , Get the improved self-attention output matrix in the sliding window at the i-th moment ,Will As the training weighted score matrix ; S16, based on the training data set Construct a principal component analysis model and repeat the process from step S12 to step S15 to obtain the fault weighted score matrix , based on the training weighted score matrix and the fault weighted score matrix Perform a second KL divergence calculation and combine it with the obtained attention statistics Get the final statistics of historical production data based on the improved self-attention mechanism KL divergence strategy ; S17, for a given impact factor , using the kernel density estimation method to estimate the final statistic Control limits .

3. The KL divergence minor fault detection method based on the improved self-attention mechanism according to claim 2 is characterized in that: In step S12, the cumulative variance contribution rate of the eigenvalue is used to determine the number of principal components of the principal component analysis model. , specifically including: Starting from the largest eigenvalue in the covariance matrix corresponding to the training data set, the variance contribution rates corresponding to the eigenvalues ​​are accumulated in sequence until the cumulative variance contribution rate reaches the preset threshold : ； represents the eigenvalue corresponding to the m'th observed variable, Before The eigenvalue corresponding to the jth observed variable in the principal component, Before The cumulative variance contribution rate corresponding to the principal component is , .

4. The KL divergence minor fault detection method based on the improved self-attention mechanism according to claim 2 is characterized in that: In step S12, the load matrix Calculate the score matrix of the training data in the principal subspace and the score matrix of the test data , including: Through the load matrix and training dataset The corresponding matrix , calculate the training data score matrix in the principal component subspace ; ； Through the load matrix and test dataset The corresponding matrix , calculate the test data score matrix in the principal component subspace ; ； for The training score, for The Test scores.

5. The KL divergence minor fault detection method based on improved self-attention mechanism according to claim 2 is characterized in that: In step S16, the combined attention statistics Get the final statistics of historical production data based on the improved self-attention mechanism KL divergence strategy , including: The attention statistics Directly assign weights, focus on important fault information, and obtain the final statistics based on the KL divergence strategy of the improved self-attention mechanism : ； in, is the weight coefficient in the weight matrix The average matrix of the weight matrix corresponding to the entire fault industrial operation process; Represent the training weighted score matrix The training weighted score and fault weighted score matrices in The weighted score of the faults in .

Images