Intermittent process dynamic fault prediction method based on adaptive temporal convolution network

Abstract

The application discloses a kind of intermittent process dynamic fault prediction methods based on adaptive timing convolution network, belong to process monitoring technical field.By constructing the coupling model of variational autoencoder and time convolution network, end-to-end joint training is carried out using offline data;During training, the hidden variable mean and logarithmic variance output by the encoding layer of variational autoencoder are used as dynamic modulation signals to inject time convolution network, so that the model perceives the global state of the process.In online monitoring, the reconstruction probability is calculated using the trained encoding layer of variational autoencoder, and incremental updating is triggered when the probability is lower than the lower limit of confidence or the log-likelihood loss exceeds the control limit;The encoding layer of variational autoencoder is frozen during updating.The updated network is used to extract multi-scale time series features, combined with attention weight to construct a weighted prediction error sequence, and fault prediction is carried out.The application improves the accuracy and timeliness of intermittent process fault prediction.

Description

Technical Field

[0001] This invention belongs to the field of intermittent process fault prediction, and relates to a method for dynamic fault prediction of intermittent processes based on adaptive temporal convolutional networks. Background Technology

[0002] Batch processes are an important production method in modern chemical, pharmaceutical, and bioengineering fields, characterized by small batches, multiple varieties, and high added value. Due to the complex physicochemical reactions, multi-stage operation switching, and dynamic differences between batches within the batch process cycle, process variables exhibit strong nonlinearity, time-varying characteristics, and multimodal features. Traditional multivariate statistical process monitoring methods (such as principal component analysis and partial least squares) assume that process data follow a Gaussian distribution and that variables have linear relationships, making it difficult to effectively capture the complex dynamic characteristics of batch processes.

[0003] In recent years, deep learning technology has received widespread attention in the field of process fault diagnosis due to its powerful nonlinear modeling capabilities. Methods based on recurrent neural networks and long short-term memory networks can model the dependencies of time series, but they suffer from the vanishing gradient problem when dealing with very long time series. Temporal convolutional networks, employing causal dilated convolution structures, can be computed in parallel and have controllable receptive fields, providing a new approach to process time series modeling. However, most existing methods assume that the dynamic characteristics of the process remain stable. When changes in operating conditions lead to model mismatch, the model needs to be retrained, making it difficult to meet the real-time requirements of online monitoring.

[0004] Furthermore, while normal operating data is readily available for intermittent processes, fault data is scarce, leading to sample imbalance issues in traditional supervised learning methods. The challenge of intermittent process fault prediction lies in how to utilize unsupervised and semi-supervised learning strategies to achieve adaptive model updates and early fault warnings. Summary of the Invention

[0005] In view of the problems of insufficient adaptability to dynamic changes in processes and rigid model update strategies in existing intermittent process fault prediction methods, a dynamic fault prediction method for intermittent processes based on adaptive temporal convolutional networks is proposed to overcome or at least partially solve the above problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a method for predicting intermittent process dynamic faults based on an adaptive temporal convolutional network, comprising the following steps: S1. Collect offline data under normal operating conditions during intermittent processes and construct a multi-scale time series sample set; S2. Obtain a coupled network consisting of a variational autoencoder and a temporal convolutional network, and input the multi-scale temporal sample set into the coupled network for end-to-end joint training; The coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives the multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. S3. Collect online process data and perform standardization processing, load the coding layer weights of the variational autoencoder, and calculate the reconstruction probability and log-likelihood loss of the online data; When the reconstruction probability is lower than the preset confidence lower limit, or the log-likelihood loss exceeds the preset control limit, the dynamic characteristics of the determination process shift significantly, triggering the model incremental update mechanism. S4. In response to the triggered incremental update mechanism of the model, process data of the current time period is collected, the coding layer weights of the variational autoencoder are frozen, and the temporal convolutional network is incrementally updated using the data of the current time period; S5. Utilize the updated temporal convolutional network to extract multi-scale temporal features of the online process data, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights; S6. Fault prediction is achieved based on the temporal evolution of the weighted prediction error sequence.

[0008] Furthermore, in step S1, the offline data is processed using the sliding window technique to construct a multi-scale time series sample set.

[0009] Furthermore, step S2, the end-to-end joint training step, also includes, The sum of prediction loss and latent space regularization loss is used as the total loss function. The parameters of the variational autoencoder and the temporal convolutional network are updated simultaneously through backpropagation. The trained temporal convolutional network is used as the baseline model, and the coding layer weights of the variational autoencoder are saved.

[0010] Further, in step S2, the mean vector and logarithmic variance vector are used as dynamic weight modulation signals and injected into the input of the temporal convolutional network, specifically including: The mean vector and logarithmic variance vector of the latent variables are used as conditional batch normalization parameters to perform affine transformation modulation on the features of the input layer of the temporal convolutional network.

[0011] Furthermore, in step S3, the confidence lower limit and the control limit are determined as follows: The multi-scale temporal sample set is input into the trained variational autoencoder, and the reconstruction probability and log-likelihood loss of all samples are calculated. The 5th percentile of the reconstruction probability is taken as the lower confidence limit, and the mean of the log-likelihood loss plus three times the standard deviation is taken as the control limit.

[0012] Furthermore, in step S4, the incremental update step of the temporal convolutional network using the current time period data employs an elastic weight consolidation algorithm. A regularization term is introduced during the update to protect network parameters important to historical tasks. Its loss function is:

[0013] in, For the prediction loss on the new subset of data; These are the temporal convolutional network parameters before the update. This represents the i-th diagonal element of the Fisher information matrix; is the regularization coefficient.

[0014] Furthermore, in step S4, the training data used for incremental updates is collected through a time-adaptive sliding window, specifically including the following steps: Calculate the intensity index of process change Δ(t):

[0015] Where M represents the most recent total number of time steps, and t represents the current time; Update the window length based on the intensity index of process change. :

[0016] in, and These are the preset minimum and maximum window lengths, respectively. The attenuation coefficient; Take all samples within a window of length W(t) forward from the current time t to form a time-series sample subset related to the current working condition, which will be used as training data for incremental updates.

[0017] Further, in step S5, the weighted prediction error sequence is obtained in the following way: Let the prediction value of the temporal convolutional network at time step i be... The actual value is residual Attention weight is The weighted prediction error sequence is then:

[0018] in, This is the weighted prediction error sequence, representing the specific mathematical form of the process health indicator, where t represents the current time index and K is the attention window length.

[0019] Furthermore, step S6, based on the temporal evolution of the weighted prediction error sequence, achieves fault prediction, specifically including: The distribution parameters of the weighted prediction error sequence are estimated using the variational Bayesian inference method, and combined with the Bayesian change point detection algorithm to achieve early warning and trend prediction of faults.

[0020] Secondly, embodiments of the present invention also disclose an intermittent process dynamic fault prediction system based on an adaptive temporal convolutional network, comprising: The data acquisition and sample construction module is used to collect offline data under normal operating conditions of intermittent processes and construct multi-scale time series sample sets. The model joint training module is used to acquire or construct a coupled network consisting of a variational autoencoder and a temporal convolutional network, and input the multi-scale temporal sample set into the coupled network for end-to-end joint training. The coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives the multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. The online monitoring and triggering module is used to collect online process data and perform standardized processing, load the coding layer weights of the variational autoencoder, and calculate the reconstruction probability and log-likelihood loss of the online data. When the reconstruction probability is lower than the preset confidence lower limit, or the log-likelihood loss exceeds the preset control limit, the dynamic characteristics of the determination process shift significantly, triggering the model incremental update mechanism. The incremental update module is used to respond to the triggered model incremental update mechanism, collect process data for the current time period, freeze the coding layer weights of the variational autoencoder, and perform incremental updates on the temporal convolutional network using the data for the current time period. The health indicator construction module is used to extract multi-scale temporal features of online process data using the updated temporal convolutional network, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights. The fault early warning module is used to predict faults based on the time-series evolution of the weighted prediction error sequence.

[0021] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method for predicting intermittent process dynamic faults based on adaptive temporal convolutional networks, which has the following beneficial effects: This invention utilizes the joint training of a variational autoencoder and a temporal convolutional network. The mean and logarithmic variance of the latent variables extracted by the variational autoencoder are used as dynamic weight modulation signals and injected into the temporal convolutional network in a conditional batch normalization manner. This enables the temporal convolutional network to adaptively adjust its feature response according to the current working condition, thereby achieving working condition adaptive temporal modeling. Compared with a simple serial combination method, this invention has a stronger dynamic adaptation capability.

[0022] After joint training is completed, the encoding layer of the variational autoencoder is used as an independent sensing module to perform unsupervised model mismatch detection using reconstruction probability. Online sensing of operating condition drift can be achieved without fault labels, solving the monitoring problem in scenarios where fault data is scarce.

[0023] This invention employs an elastic weight consolidation algorithm to achieve incremental learning of temporal convolutional networks. It measures parameter importance through the Fisher information matrix, protecting important parameters while updating the model to adapt to new operating conditions, effectively avoiding catastrophic forgetting. At the same time, it freezes the coding layer of the variational autoencoder to ensure the stability of the reference system under normal operating conditions.

[0024] This invention combines an attention mechanism and a Bayesian change point detection algorithm to construct a weighted prediction error sequence as a health indicator. The attention mechanism automatically focuses on the time step most relevant to the fault evolution, enabling probability inference and early warning of the fault occurrence time, and providing operators with more accurate fault response timing. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0026] Figure 1 The overall flowchart of the method provided by the present invention.

[0027] Figure 2 This is a schematic diagram of the joint framework of temporal convolutional network and variational autoencoder provided by the present invention.

[0028] Figure 3 This is the result of the method of the present invention for fault prediction in the penicillin fermentation process.

[0029] Figure 4Box plot of predictive index statistics for test batch data provided in this embodiment of the invention. Detailed Implementation

[0030] The following will refer to the appendices in the embodiments of the present invention. Figure 1 -Appendix Figure 4 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] A dynamic fault prediction method for intermittent processes based on adaptive temporal convolutional networks is presented. Since penicillin fermentation is a typical intermittent process, this invention uses the penicillin fermentation process as an example to illustrate the specific implementation method.

[0032] Penicillin fermentation is a typical time-varying batch process involving complex biological and chemical changes from raw materials to the final product. Numerous control loops and variables exist during fermentation, frequently leading to malfunctions that severely impact production performance and product quality. Industrial-scale penicillin fermentation platforms, in addition to modeling necessary online and offline variables, consider the growth, metabolism, and degradation of penicillin during large-scale fermentation. Multiple control strategies are specified to reduce system fluctuations and maximize penicillin yield. This study selected 50 batches of data under different control strategies, with a fermentation reaction time of 230 hours, where data were collected every 12 minutes. Twenty key process variables were selected as offline data, as shown in Table 1.

[0033] Table 1. Description of process variables during penicillin fermentation.

[0034] like Figure 1 The diagram shows a flowchart of the intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks disclosed in this invention. The specific steps are as follows: Step S1: Collect offline data and construct a multi-scale time series sample set Collect offline data under normal operating conditions. Because the batch process exhibits different dynamic characteristics at different reaction stages (such as the initial cell growth stage of fermentation, the intermediate product synthesis stage, and the final decay stage), it is difficult to capture the characteristics of each stage simultaneously with a single window length.

[0035] To this end, the present invention uses multiple sliding windows of different lengths to slice offline data, extracting local dynamic feature samples at short time scales and global trend feature samples at long time scales respectively, forming a sample set with multi-scale receptive fields, namely a multi-scale time series sample set.

[0036] Step S2: Jointly train the variational autoencoder and the temporal convolutional network In this step, a coupled network consisting of a variational autoencoder and a temporal convolutional network is constructed, and a multi-scale temporal sample set is input into the coupled network for end-to-end joint training. Among them, the coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives a multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. The sum of prediction loss and latent space regularization loss is used as the total loss function. The parameters of the variational autoencoder and the temporal convolutional network are updated simultaneously through backpropagation. The trained temporal convolutional network is used as the baseline model, and the coding layer weights of the variational autoencoder are saved. A variational autoencoder (VAE) consists of an encoder network and a decoder network. The core idea of ​​a variational autoencoder is to assume the existence of a low-dimensional latent variable z that controls the generation process of high-dimensional observation data x. In a standard variational autoencoder, the latent variable z is assumed to follow a standard normal distribution prior p(z) = N(0, I).

[0037] The role of the encoding network (encoding layer) is to infer the posterior distribution p(z|x) of the latent variable z, given input data x. Since this posterior distribution is usually unsolvable, the variational autoencoder employs variational inference, introducing a neural network (i.e., the encoding network) to approximate this posterior distribution, denoted as p(z|x). And assume that the approximate posterior distribution is a diagonal Gaussian distribution:

[0038] in, It is the mean vector. It is the variance vector. This is the logarithmic variance vector. `diag` represents a diagonal matrix.

[0039] In other words, the encoding network in a variational autoencoder (VAE) can map the input data to the mean and log-variance parameters of the latent variable distribution, and the decoding network reconstructs the original input from the latent variables.

[0040] Specifically, regarding the process data matrix X∈R in this invention T×D Where T is the time step (230 hours × 5 sampling points / hour = 1150 sampling points), and D is the variable dimension (20 main process variables). The encoding network encodes the input data x for each time step. t ∈R DMapped to the mean vector of the latent variable distribution ∈R H Sum of logarithmic variance vector ∈R H , where H is the dimension of the latent variable (H=8 in this embodiment).

[0041] Mean vector Sum of logarithmic variance vector It contains global state information of the process data at the current moment (such as the current reaction stage and operating condition). This invention uses it as a dynamic weight modulation signal to inject into the input of the temporal convolutional network (TCN), enabling the TCN to adaptively adjust its feature extraction method according to the current process state.

[0042] The specific injection method is Conditional Batch Normalization, and its core calculation process is as follows: Let h∈R be the intermediate feature maps of a certain batch in the input layer of a temporal convolutional network. B×C×L Where B is the batch size, C is the number of channels, and L is the sequence length. The calculation for regular batch normalization is as follows:

[0043] in, and The mean and variance of the current batch. and These are learnable scaling and offset parameters.

[0044] In conditional batch normalization and Instead of fixed parameters, the modulated signal output from the VAE coding layer is dynamically generated through a small fully connected network: ;

[0045] in, and As a fully connected mapping network, the latent variable parameters are mapped to modulation vectors that match the number of channels in the feature map. Therefore, the feature extraction response of the temporal convolutional network will dynamically change with the process operating conditions, achieving adaptive modeling of the operating conditions.

[0046] In this embodiment, the encoder network and the decoder each contain 3 hidden layers, with 128, 64 and 32 neurons in each layer, respectively, and the hidden variable dimension H=8.

[0047] The temporal convolutional network adopts a causal dilated convolutional structure, with the dilation factor increasing exponentially with the number of layers, so that the receptive field covers the entire fermentation cycle. In this embodiment, the dilation factor of the temporal convolutional network is set to 2, the kernel size is set to 3, and the number of network layers is set to 6, so that the receptive field covers about 250 time steps.

[0048] The forward propagation process of the entire coupled network is as follows: the original input X is simultaneously fed into two paths: one enters the VAE encoding layer to extract latent variable parameters (mean and log-variance), and the other serves as the input to the temporal convolutional network TCN; the modulated signal output from the VAE encoding layer is injected into the input of the TCN; after receiving the original temporal data and the modulated signal, the TCN performs multi-scale temporal feature extraction and outputs the predicted value. .

[0049] The total loss function is:

[0050] in: The mean squared error between the predicted value and the true value output by the temporal convolutional network; , which is the KL divergence between the latent variable distribution of the variational autoencoder and the standard normal distribution, used to constrain the structured distribution of the latent variable space; The weighting coefficients are used to adjust the trade-off between reconstruction accuracy and latent space regularization.

[0051] During training, the parameters of the variational autoencoder's coding layer are updated simultaneously through backpropagation, with the total loss function as the objective. The decoding layer parameters θ and all trainable parameters of the temporal convolutional network.

[0052] After training, the temporal convolutional network is used as the baseline model (i.e., the standard reference model representing the dynamic characteristics of the process under normal operating conditions). Simultaneously, the weight parameters of the variational autoencoder layer are saved (for subsequent online monitoring), and the reconstruction probability and log-likelihood loss values ​​of all samples on the training set are recorded. In this invention, the reconstruction probability is defined as the log-likelihood expectation of the reconstructed data under the posterior distribution of the latent variables, estimated using Monte Carlo sampling, and the confidence interval under normal operating conditions is determined using kernel density estimation.

[0053] Step S3: Online monitoring and incremental update triggering Collect online process data and perform standardization (the mean and variance used for standardization are derived from the statistical values ​​of the offline training set to ensure the consistency of data distribution).

[0054] Load the pre-trained variational autoencoder encoding layer weights (which remain fixed), input online data into the encoding layer, and calculate the reconstruction probability and log-likelihood loss of the online data.

[0055] The reconstruction probability is defined as the expected log-likelihood of the reconstructed data under the posterior distribution of the latent variables, and is estimated using Monte Carlo sampling.

[0056] Where L is the number of samples (L=100 in this embodiment). The posterior distribution output from the coding layer Latent variable samples obtained from sampling The reconstruction distribution defined for the decoder.

[0057] The confidence lower limit and control limit are determined as follows: The multi-scale time-series sample set from the offline training set is input into the trained variational autoencoder, and the reconstruction probability and log-likelihood loss of all samples are calculated. The 5th percentile of the reconstruction probability is taken as the confidence lower limit (i.e., under normal operating conditions, the reconstruction probability of 95% of samples should be higher than this value), and the mean of the log-likelihood loss plus three standard deviations is taken as the control limit (based on the 3σ criterion, under normal operating conditions, 99.7% of samples should be within this range).

[0058] During online monitoring, if the reconstruction probability at the current moment is lower than the confidence limit, or the log-likelihood loss exceeds the control limit, it indicates that the latent space distribution of the current process data has significantly deviated from the normal operating condition distribution. This indicates that the dynamic characteristics of the process have significantly shifted, triggering the model incremental update mechanism.

[0059] Step S4: Elastic Weight Consolidation Incremental Update S41 Incremental Update Triggering and Data Acquisition In response to the triggered incremental model update mechanism, the system needs to learn from the process data under the new operating conditions. However, using all historical data indiscriminately may introduce noise information that is irrelevant to the current operating conditions.

[0060] Therefore, this invention employs a time-adaptive sliding window strategy to collect training data.

[0061] Define the process change intensity index Δ(t):

[0062] The first term represents the Euclidean distance between data changes at adjacent time steps, the second term represents the difference in the gradient of the loss function, and α is the balance coefficient. This metric comprehensively reflects the drastic changes in the process at both the data level and the model optimization level.

[0063] In another optimized embodiment, considering the average degree of change within a historical window, the mean of the L2 norm of the data changes in adjacent time steps within the sliding window can also be used as an indicator of the intensity of process change. Taking the overall degree of change in the most recent M time steps as an example, the intensity of process change Δ(t) is defined as follows:

[0064] Where M represents the most recent total time step and t represents the current time, in this embodiment, the impact of single-step random fluctuations is smoothed by averaging the M differences.

[0065] The window length W(t) is dynamically updated according to the following formula:

[0066] in, and These are the preset minimum and maximum window lengths (set to 50 and 200 respectively in this embodiment). The decay coefficient is used. During the dynamic update of the window length, an exponential decay mapping is employed to represent the intensity index of process change. Mapped to [ Within the interval, when When the value approaches zero (the process is stationary), exp(0) = 1, and the window length W(t) approaches its maximum value. The window can be extended appropriately to increase the amount of data; when When the change is large (the process changes drastically), As the time approaches 0, the window length W(t) approaches its minimum value. The shorter the window length, the more focused the selected samples are on the current operating conditions.

[0067] If the continuous change in window length during the process is not considered, the window length W(t) can be updated by inversely proportional truncation, as shown in the following formula:

[0068] Finally, all samples within a window of length W(t) preceding the current time t are selected to form the time series sample subset most relevant to the current operating condition. , as training data for incremental updates.

[0069] Incremental update mechanism of S42 elastic weight consolidation algorithm Incremental updates face the challenge of catastrophic forgetting: if the model is trained directly with new data, it will gradually forget the knowledge learned under the original normal operating conditions. This invention uses the Elastic Weight Consolidation (EWC) algorithm to solve this problem.

[0070] The core idea of ​​the Elastic Weight Consolidation algorithm is to impose significant constraints on important parameters from historical tasks when updating model parameters, making them less prone to change, while allowing for larger adjustments to less important parameters. Parameter importance is determined by the diagonal elements of the Fisher Information Matrix. To measure, The larger the value, the more important the parameter is for the prediction accuracy of historical tasks.

[0071] The loss function for incremental updates is:

[0072] in, , which is the prediction loss (mean squared error) on the new data subset. The parameters of the temporal convolutional network before the update (i.e., the baseline model or the parameters after the last update); Let be the i-th diagonal element of the Fisher information matrix, obtained by calculating the expectation of the squared gradient on historical training data before updating, and quantify the parameter. Importance to historical missions; This is the regularization coefficient (set to 0.1 in this embodiment), which controls the regularization strength.

[0073] The optimization objective of this loss function is to minimize the prediction error of new data while keeping the important parameters as close to their old values ​​as possible. The nearby environment allows for adaptation to new working conditions while preserving knowledge of normal working conditions.

[0074] It is particularly important to emphasize that the coding layer weights of the variational autoencoder are completely frozen during the incremental update process and do not participate in any parameter updates. This is because the VAE coding layer establishes a latent space distribution benchmark based on the original normal operating condition. If it were updated along with the TCN, this benchmark would drift, causing the reconstruction probability calculation in the subsequent online monitoring phase to lose its reference value. Freezing the VAE coding layer ensures that the reference frame of "normal operating condition" remains stable at all times.

[0075] After the incremental update is completed, the updated TCN parameters will be used as the new baseline to continue the subsequent online monitoring and prediction tasks.

[0076] Step S5: Construct a weighted prediction error sequence The updated temporal convolutional network is used to extract multi-scale temporal features of online process data, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights.

[0077] Let the prediction value of the temporal convolutional network at time step i be... The actual value is residual The residuals themselves reflect the prediction bias at the current moment, but different time steps have different importance in indicating the health status of the process (for example, small deviations in some stages may indicate serious failure evolution, while deviations in other stages may just be normal fluctuations). To address this, this invention introduces an attention mechanism to automatically learn the importance weights of each time step.

[0078] Attention weight The calculation method is as follows: For each time step feature map output by the last residual block of the temporal convolutional network... The importance score for each time step is calculated through a fully connected layer. Then, after softmax normalization:

[0079] Where v represents the weight vector, tanh represents the hyperbolic tangent activation function, and W is the weight matrix. Let represent the i-th input feature, b represent the bias vector, K represent the attention window length, and the physical meaning of the attention weight is: the time step that is most relevant to the current process state and best reflects the trend of health deterioration receives a higher weight.

[0080] Finally, the weighted prediction error sequence is as follows:

[0081] This refers to the specific mathematical form of the health indicators in the process, which amplifies subtle abnormal trends and suppresses irrelevant random fluctuations.

[0082] Step S6: Bayesian Variable Point Detection Fault Early Warning Based on weighted prediction error sequence The temporal evolution pattern is analyzed, and the probability distribution parameters are estimated using variational Bayesian inference. Combined with the Bayesian change point detection algorithm, early warning and trend prediction of faults are achieved.

[0083] The core idea of ​​Bayesian change point detection is to view the health indicator sequence as a series of segmented steady-state processes, where the data in each steady-state segment follows the same distribution. When the distribution parameters change abruptly between different segments, the point of change is the potential time when a fault occurs.

[0084] Specifically, the Bayesian change point detection algorithm is used to analyze the weighted prediction error sequence. Perform online monitoring and define the runtime. This represents the number of time steps from the current time t to the previous change point. It is a non-negative discrete random variable.

[0085] Run length from the previous moment Transition to the current moment The probability is:

[0086] in, "Dangerous function" indicates a runtime of... The probability of a change point occurring at that time.

[0087] State transition probability of runtime During the calculation, if no point changes in the current step, then If a change point occurs, then ; The observation likelihood (representing the conditional probability of the current observation given historical observations and the current runtime). Joint distribution of current running length and all observed sequences ( The recursive probability is:

[0088] in, This represents the weighted prediction error observation sequence from time 1 to time t. The distribution parameters are calculated from the variational Bayesian inference estimate to obtain the observation likelihood.

[0089] The marginal probability of the observed sequence at the current time is:

[0090] The posterior probability distribution of the runtime at the current moment is:

[0091] The posterior probability of the change point is The probability is:

[0092] when When the preset alarm threshold (set to 0.7 in this embodiment) is exceeded, the current time is determined to be the time when a potential fault occurs, and a fault warning signal is triggered.

[0093] Meanwhile, based on the variational Bayesian inference of the estimated distribution parameter change trend, the degradation trajectory of health indicators in future time steps can be extrapolated, enabling trend prediction of fault severity and providing a reference for operators. For example, if the estimated distribution mean shows a monotonically increasing trend and the rate of increase is accelerating, it indicates that the fault is rapidly deteriorating.

[0094] The method disclosed in this invention can be implemented by a computer system, which includes: The data acquisition and sample construction module is used to collect offline data under normal operating conditions of intermittent processes and construct multi-scale time series sample sets. The model joint training module is used to acquire or construct a coupled network consisting of a variational autoencoder and a temporal convolutional network, and input the multi-scale temporal sample set into the coupled network for end-to-end joint training. The coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives the multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. The online monitoring and triggering module is used to collect online process data and perform standardized processing, load the coding layer weights of the variational autoencoder, and calculate the reconstruction probability and log-likelihood loss of the online data. When the reconstruction probability is lower than the preset confidence lower limit, or the log-likelihood loss exceeds the preset control limit, the dynamic characteristics of the determination process shift significantly, triggering the model incremental update mechanism. The incremental update module is used to respond to the triggered model incremental update mechanism, collect process data for the current time period, freeze the coding layer weights of the variational autoencoder, and perform incremental updates on the temporal convolutional network using the data for the current time period. The health indicator construction module is used to extract multi-scale temporal features of online process data using the updated temporal convolutional network, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights. The fault early warning module is used to predict faults based on the time-series evolution of the weighted prediction error sequence.

[0095] Examples of failure prediction results in the penicillin fermentation process of this invention are as follows: Figure 3 As shown in the figure, the box plot of the predictive index for multiple test batches of data for different faults is as follows: Figure 4 As shown.

[0096] The network model proposed in this invention provides accurate fault prediction, fully considering the time-varying characteristics of intermittent processes. It utilizes a variational autoencoder to extract deep latent variable features and establishes a model mismatch judgment index based on reconstruction probability. A temporal convolutional network is used to extract multi-scale temporal features, constructing process health indicators. An elastic weight consolidation algorithm is used for incremental updates, combined with Bayesian change point detection to achieve trend prediction and early warning. When the monitoring model is mismatched and needs updating, an adaptive window is used to select the update dataset, and an elastic weight consolidation strategy is employed to update the model to adapt to dynamic time-varying characteristics, improving the performance of fault prediction and the timeliness of early warning. It monitors the fault status of complex intermittent processes in real time and accurately predicts fault trends.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for predicting intermittent process dynamic faults based on adaptive temporal convolutional networks, characterized in that, Includes the following steps: S1. Collect offline data under normal operating conditions during intermittent processes and construct a multi-scale time series sample set; S2. Obtain a coupled network consisting of a variational autoencoder and a temporal convolutional network, and input the multi-scale temporal sample set into the coupled network for end-to-end joint training; The coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives the multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. S3. Collect online process data and perform standardization processing, load the coding layer weights of the variational autoencoder, and calculate the reconstruction probability and log-likelihood loss of the online data; When the reconstruction probability is lower than the preset confidence lower limit, or the log-likelihood loss exceeds the preset control limit, the dynamic characteristics of the determination process shift significantly, triggering the model incremental update mechanism. S4. In response to the triggered incremental update mechanism of the model, process data of the current time period is collected, the coding layer weights of the variational autoencoder are frozen, and the temporal convolutional network is incrementally updated using the data of the current time period; S5. Utilize the updated temporal convolutional network to extract multi-scale temporal features of the online process data, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights; S6. Fault prediction is achieved based on the temporal evolution of the weighted prediction error sequence.

2. The intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S1, the sliding window technique is used to process offline data and construct a multi-scale time series sample set.

3. The intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks according to claim 1, characterized in that, Step S2, the end-to-end joint training step, also includes, The sum of prediction loss and latent space regularization loss is used as the total loss function. The parameters of the variational autoencoder and the temporal convolutional network are updated simultaneously through backpropagation. The trained temporal convolutional network is used as the baseline model, and the coding layer weights of the variational autoencoder are saved.

4. The intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S2, the mean vector and log-variance vector are used as dynamic weight modulation signals and injected into the input of the temporal convolutional network, specifically including: The mean vector and logarithmic variance vector of the latent variables are used as conditional batch normalization parameters to perform affine transformation modulation on the features of the input layer of the temporal convolutional network.

5. The method for predicting intermittent process dynamic faults based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S3, the confidence lower limit and the control limit are determined as follows: The multi-scale temporal sample set is input into the trained variational autoencoder, and the reconstruction probability and log-likelihood loss of all samples are calculated. The 5th percentile of the reconstruction probability is taken as the lower confidence limit, and the mean of the log-likelihood loss plus three times the standard deviation is taken as the control limit.

6. The intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S4, the incremental update step of the temporal convolutional network using current time period data employs an elastic weight consolidation algorithm. A regularization term is introduced during the update to protect network parameters important to historical tasks. Its loss function is: in, For the prediction loss on the new subset of data; These are the temporal convolutional network parameters before the update. This represents the i-th diagonal element of the Fisher information matrix; This is the regularization coefficient.

7. The method for predicting intermittent process dynamic faults based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S4, the training data used for incremental updates is collected through a time-adaptive sliding window, specifically including the following steps: Calculate the intensity index of process change Δ(t): Where M represents the most recent total number of time steps, and t represents the current time; Update the window length based on the intensity index of process change. : in, and These are the preset minimum and maximum window lengths, respectively. The attenuation coefficient; Take all samples within a window of length W(t) forward from the current time t to form a time-series sample subset related to the current working condition, which will be used as training data for incremental updates.

8. The intermittent process dynamic fault prediction method based on adaptive temporal convolutional networks according to claim 1, characterized in that, In step S5, the weighted prediction error sequence is obtained in the following way: Let the prediction value of the temporal convolutional network at time step i be... The actual value is residual Attention weight is The weighted prediction error sequence is then: in, This is the weighted prediction error sequence, representing the specific mathematical form of the process health indicator, where t represents the current time index and K is the attention window length.

9. The method for predicting intermittent process dynamic faults based on adaptive temporal convolutional networks according to claim 1, characterized in that, Step S6, based on the temporal evolution of the weighted prediction error sequence, achieves fault prediction, specifically including: The distribution parameters of the weighted prediction error sequence are estimated using variational Bayesian inference, and combined with the Bayesian change point detection algorithm to achieve early warning and trend prediction of faults.

10. A dynamic fault prediction system for intermittent processes based on an adaptive temporal convolutional network, characterized in that, include: The data acquisition and sample construction module is used to collect offline data under normal operating conditions of intermittent processes and construct multi-scale time series sample sets. The model joint training module is used to acquire or construct a coupled network consisting of a variational autoencoder and a temporal convolutional network, and input the multi-scale temporal sample set into the coupled network for end-to-end joint training. The coding layer of the variational autoencoder extracts features from the multi-scale temporal sample set, outputs the mean vector and log-variance vector of the latent variables, and uses the mean vector and log-variance vector as dynamic weight modulation signals, which are then injected into the input of the temporal convolutional network. The temporal convolutional network receives the multi-scale temporal sample set and the dynamic weight modulation signal, and performs temporal feature extraction and prediction output. The online monitoring and triggering module is used to collect online process data and perform standardized processing, load the coding layer weights of the variational autoencoder, and calculate the reconstruction probability and log-likelihood loss of the online data. When the reconstruction probability is lower than the preset confidence lower limit, or the log-likelihood loss exceeds the preset control limit, the dynamic characteristics of the determination process shift significantly, triggering the model incremental update mechanism. The incremental update module is used to respond to the triggered model incremental update mechanism, collect process data for the current time period, freeze the coding layer weights of the variational autoencoder, and perform incremental updates on the temporal convolutional network using the data for the current time period. The health indicator construction module is used to extract multi-scale temporal features of online process data using the updated temporal convolutional network, calculate the prediction residuals at each time step, and construct a weighted prediction error sequence by combining attention weights. The fault early warning module is used to predict faults based on the time-series evolution of the weighted prediction error sequence.

Images