Industrial manufacturing process fault self-healing method

Abstract

In order to solve the technical problem that the existing fault repair method cannot realize fault self-recovery from the fault source, the present application provides an industrial manufacturing process fault self-recovery method. The present application comprehensively extracts data features from manufacturing process data by using an automatic encoder improved time convolution network, and then constructs a causal relationship reasoning graph of process variables; then the multi-head attention mechanism is introduced into the causal relationship reasoning graph, the variable weight is recalculated, the interpretability of the deep learning method is enhanced, and when the multi-element time series data is used, the multi-head attention mechanism can directly measure the relationship of the variables; then the scoring rules of the root node S root are given, the influence of time lag and the gain of causal relationship are considered, and the root cause is quantitatively judged; finally, the adaptive biological immune reinforcement learning method is used for self-recovery of the root cause of the fault.

Description

Technical Field

[0001] This invention relates to a method for self-healing faults in industrial manufacturing processes. Background Technology

[0002] Intelligent industrial systems represent a crucial direction for the future development of industry and are essential for the long-term safe and reliable operation of industrial systems. In automated industrial processes, sensors installed on industrial equipment can collect and transmit various data from actual production in a timely manner, providing a data foundation for intelligent industrial process monitoring.

[0003] Intelligent industrial process monitoring is a closed loop consisting of multiple parts, including fault detection, fault isolation, fault identification, and fault recovery. Among these, fault isolation and fault identification can be categorized as fault diagnosis in a narrow sense, and fault diagnosis and fault detection together are referred to as process monitoring.

[0004] Many scholars have conducted research on process monitoring in intelligent manufacturing and applied it to various fields, such as metallurgy, chemical engineering, petrochemicals, and steelmaking. However, in actual industrial manufacturing processes, the level of automated fault repair remains low when faults occur. Most repairs rely on manual methods to fix obvious faults, which is not only costly, difficult, and extremely dangerous, but also addresses the root cause of the fault by only addressing the surface and repairing readily observable problems.

[0005] There are some automated fault repair methods available, but they also cannot deduce the root cause of the fault and solve the problem at its source.

[0006] Therefore, how to achieve fault self-healing from the source of the fault is an urgent problem to be solved in the intelligent manufacturing process. Summary of the Invention

[0007] In order to solve the technical problem that existing fault repair methods cannot achieve fault self-healing from the source of the fault, this invention provides a fault self-healing method for industrial manufacturing processes.

[0008] The technical solution of this invention is:

[0009] The self-healing method for industrial manufacturing process faults is unique in that it includes the following steps:

[0010] Step 1: Collect manufacturing variable data related to the production process through simulation. After introducing known fault types for labeling, divide the data into normal production dataset and faulty production dataset, and use them as training set and test set respectively.

[0011] Step 2: Preprocess the manufacturing variable data in the training and test sets by removing non-numeric characters and standardizing them;

[0012] Step 3: Construct the TCN-VAE causal inference network model and set its initial parameters. Train the network model using the data in the training set after preprocessing in Step 2. During training, update and iterate the network parameter settings through the loss function and optimization function, and test the model using the test set until the set requirements are met, at which point the model training terminates. The initial parameters include input format, receptive field, convolutional kernel size, convolutional layer depth, maximum length of process variable data, model optimization function, activation function, number of iterations, loss function, and network layer structure.

[0013] Step 4: Input the manufacturing variable data collected in actual production into the TCN-VAE causal reasoning network model in Step 3. The TCN-VAE causal reasoning network model outputs a causal relationship reasoning graph.

[0014] Step 5: In the causal relationship inference graph obtained in step 4, a multi-head attention mechanism is introduced to calculate and update the weights of each feature graph, and irrelevant causal relationships in the causal relationship inference graph are eliminated by time feature gradient error, so as to further distinguish the direct and indirect causes of the failure.

[0015] Step 6: Based on the causal reasoning graph obtained after eliminating irrelevant causal relationships in Step 5, and the distinction between direct and indirect causes in Step 5, establish and quantitatively determine the root cause of the fault by considering the impact of time delay and causal relationship gain according to the scoring rules of the root node.

[0016] Step 7: Use adaptive biological immune reinforcement learning to self-heal the root cause of the fault obtained in Step 5.

[0017] Furthermore, the TCN-VAE causal inference network model described in step 3 includes N parallel-connected, independently operating, and structurally identical TCN channels, as well as a TCN-VAE encoder and a TCN-VAE decoder; the outputs of the N TCN channels are connected to the input of the TCN-VAE encoder, and the output of the TCN-VAE encoder is connected to the input of the TCN-VAE decoder; the N TCN channels are used for convolutional extraction of temporal features; the TCN-VAE encoder is used to reconstruct time-series features; and the TCN-VAE decoder is used to decode the time-series features reconstructed by the TCN-VAE encoder and output a causal inference graph.

[0018] Furthermore, when reconstructing time series features, the TCN-VAE encoder uses KL divergence to measure the difference between the reconstructed features and the original features.

[0019] Furthermore, the method for introducing a multi-head attention mechanism to calculate each feature map in the causal reasoning graph in step 5 is as follows:

[0020] The formula for establishing an attention mechanism from the input feature map through three subspaces is as follows:

[0021]

[0022] Where Q = W Q X θ K = W K X θ V = W V X θ d k Represents the input feature map X θ The dimension is used to scale the dot product to prevent vanishing, AT(·) denotes the attention mechanism, and W Q W K W V Let Q be the subspace projection matrix, Q and K be matrices representing the similarity of feature maps, V be the matrix obtained by weighting Q and K as attention weights, and A be the attention coefficient matrix, representing the degree of influence between manufacturing process variables.

[0023] Furthermore, the root node scoring rule established in step 6 is as follows: calculate the score of each root node, and take the root node with the highest score as the root cause of the fault.

[0024] The formula for calculating the fraction of the root node is:

[0025]

[0026] Where Num represents the maximum number of generation of the root node, M represents the number of iterations of the child nodes, and k n,m μ represents the causal gain of the m-th child node in the n-th iteration. n S represents the weight in the nth iteration. root The score of the root node.

[0027] Furthermore, step 7 specifically includes:

[0028] Step 7.1 Construct a reinforcement learning fault self-healing model based on the effector T cell immune process, set initial values ​​for reinforcement learning parameters, and perform self-learning iterations on the reinforcement learning parameters; the fault self-healing model includes T helper cells, antigen-presenting cells, and activated effector cells; when a fault occurs, antigen-presenting cells are used to engulf pathogens and present multiple faults in different fault modes of the manufacturing process variables; T helper cells are used to extract fault state information in the manufacturing process from the fault modes presented by antigen-presenting cells, and enable the activated effector cells to perform a self-healing operation response to the fault state information, clearing the fault from the T helper cells with a certain probability; the self-learning is achieved by modulating the distribution of T helper cell clone size;

[0029] Step 7.2 Input the root cause of the fault obtained in Step 6 and the set of fault manufacturing process variables related to the root cause collected during the production process into the reinforcement learning fault self-healing model constructed in Step 7.1 for self-healing repair.

[0030] Furthermore, the reinforcement learning parameters of the fault self-healing model include:

[0031] The self-healing response generated by activated effector cells;

[0032] The fault self-healing transfer probability is used to modulate the probability of activated effector cells being activated or inactivated by various fault modes. When a fault is detected, the activated effector cells repair the fault according to the fault self-healing transfer probability.

[0033] The feedback reward function is used to characterize the effectiveness of the self-healing response generated by activated effector cells in responding to the fault. When the self-healing response generated by activated effector cells deviates from the most effective active self-healing pattern, the immune system loses the reward; when the self-healing response generated by activated effector cells matches the most effective active self-healing pattern, the immune system receives the highest reward.

[0034] The random transfer probability distribution strategy of the T helper cell population is used to realize the interaction between manufacturing process variables, so that the fault self-healing model can make adaptive repairs to faults.

[0035] The beneficial effects of this invention are:

[0036] 1. This invention infers the root cause of a fault through mapping between complex event sequences, and utilizes an adaptive biological immune reinforcement learning method for fault self-healing, ultimately resolving the fault from its source and achieving fault self-healing.

[0037] 2. Existing root cause diagnosis research mostly adopts a data-driven approach, relying on analyzing the causal relationships between variables to form a causal graph, thereby discovering the evolution process of faults and deeply mining the interaction information between variables. Currently, manufacturing process data faces problems such as multi-scale, temporal characteristics, and nonlinearity. Existing data-driven root cause inference methods are mostly black-box, unable to explain the relationships between variables, and lack sufficient feature extraction and analysis, resulting in low accuracy in inferring the fault evolution process. This invention's root cause diagnosis method uses a temporal convolutional network (TCN) improved from a variational autoencoder (VAE) to comprehensively extract data features from manufacturing process data, thereby constructing a causal relationship inference graph of process variables. Then, a multi-head attention mechanism is introduced into the causal relationship inference graph to recalculate variable weights, which enhances the interpretability of deep learning methods. Furthermore, for multivariate time series data, the multi-head attention mechanism can directly numerically measure the relationships between variables. Finally, considering that there are usually multiple causal transmission paths in actual manufacturing processes, it is impossible to trace from multiple root nodes S... root To determine the root cause, this invention provides the root node S. root The scoring rules take into account the effects of time delay and causal relationships to increase the gain, thereby enabling quantitative judgment of the root cause and providing a basis for subsequent fault self-healing.

[0038] 3. In industrial process monitoring, when inconsistencies are found between operational data and normal manufacturing data, such as anomalies, outliers, inconsistent observations, and unexpected situations in various application systems, fault self-healing can automatically optimize these data to return to normal. Vertebrate adaptive immunity is a complex adaptive system, and fault self-healing in intelligent industrial manufacturing shares some similarities with vertebrate immune repair. Some studies, inspired by the biological immune system, have applied this approach to fault self-healing in industrial manufacturing processes. However, most existing fault self-healing methods, when considering the self-healing process of the biological immune system, focus on simple static learning, lacking a flexible embedding and a dynamic environment for continuous learning. This invention establishes a reinforcement learning-based fault self-healing model based on the adaptive immune process of effector T cells. First, T helper cells extract information about the fault state during manufacturing from the antigen presentation patterns of antigen-presenting cells (APCs). Then, T helper cells regulate the interactions between effector fault variables by secreting different types of cytokines. In the network model, the T helper cell population is constructed as an intermediate layer between the fault mode and the activated effector cell mode. When a fault occurs, APCs can engulf pathogens and present multiple faults in different fault modes. T helper cells recognize the fault modes and enable effector cells to make a self-healing response. Therefore, this invention can not only repair root cause faults, but also dynamically repair other faults caused by root cause faults.

[0039] 4. The immune repair process based on effector T cells in this invention is more in line with the principles of reinforcement learning and can better adapt to the control process of complex manufacturing systems. Attached Figure Description

[0040] Figure 1 It is an improved temporal convolutional network diagram for autoencoders.

[0041] Figure 2 It is a causal relationship reasoning diagram.

[0042] Figure 3 This is a diagram showing the relationship between adaptive biological immune activation and reinforcement learning.

[0043] Figure 4 It is a dynamic process diagram of the self-healing transition probability between normal and fault states.

[0044] Figure 5 This is a diagram showing the self-healing result of the root cause of the fault.

[0045] Figure 6 It is a graph of the average reward function.

[0046] Figure 7 This is a Sobol sensitivity analysis index distribution chart.

[0047] Figure 8It is a 3D visualization of the fault self-healing result. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below in conjunction with the accompanying drawings.

[0049] The self-healing method for industrial manufacturing process faults provided by this invention includes the following steps:

[0050] Step 1: Root cause diagnosis;

[0051] Step 1.1 Data Collection;

[0052] Taking the Eastman Tennessee and the industrial manufacturing process of continuous stirred tank reactor as an example, data is collected by simulating the manufacturing system. That is, after building a simulation model of the manufacturing system, the environment settings parameters (including at least sampling period, sampling length, output format, runtime, and disturbance factor) are configured using the MATLAB Simulink simulation toolbox and then run to perform platform simulation and collect data.

[0053] For the Eastman manufacturing process in Tennessee, the simulation collected data including manufacturing variables related to the production process. Twenty-one known fault types (derived from historical fault data) were introduced into these variables and categorized. This resulted in a normal production dataset and a faulty production dataset. The normal production dataset was used as the training set with a size of 480*52, and the faulty production dataset was used as the test set with a size of 960*52, resulting in one normal training set and 21 faulty test sets. The sizes of the training and test sets are shown here as examples; in practice, they can be manually set according to actual needs.

[0054] For the manufacturing process of a continuous stirred tank reactor, the simulation data collected includes manufacturing process variables related to the production process. Ten known fault types (derived from historical fault data) are introduced into these variables for labeling and classification, resulting in a normal production dataset and a fault-prone production dataset. The normal production dataset is used as the training set with a size of 500*52, and the fault-prone production dataset is used as the test set with a size of 960*52, resulting in one normal data training set and ten fault-prone test sets. The sizes of the training and test sets are only examples; in actual operation, they can be manually set according to specific needs.

[0055] Assume the manufacturing process variables collected by the simulation are x i x i ∈R m , i = 1, 2, ..., N, R mLet R be the set of manufacturing process variables, and N be the number of manufacturing process variables. When the model input is the training set, R... m x represents the set of manufacturing process variables in the training set. i Represents the manufacturing process variables in the training set; when the model input is the test set, R... m x represents the set of manufacturing process variables in the test set. i This represents the manufacturing process variables in the test set.

[0056] The fault test set introduces a fault at the nth sample (e.g., the 161st sample) for simulation. After the fault is introduced, it will propagate to the remaining variables over time, thus generating a fault propagation graph that needs to be detected.

[0057] Step 1.2 Data preprocessing;

[0058] Step 1.2.1 Determine whether there are special characters and stop characters or other non-numeric characters (such as English letters, / / , *, punctuation marks, etc.) in the training set and test set obtained in Step 1.1. If so, delete these non-numeric characters.

[0059] Step 1.2.2 Standardize the data in the training and test sets processed in Step 1.2.1 according to the following data standardization formula to ensure the uniformity of data dimensions in the input TCN-VAE causal inference network model and fault self-healing model, and to prevent overfitting and peak performance in the subsequently built TCN-VAE causal inference network model and fault self-healing model. The data standardization formula is as follows:

[0060]

[0061] Among them, X nor X represents the result after data standardization, and X represents the original dataset before data standardization. mean X represents the mean of data belonging to the same original dataset. std This represents the standard deviation of data belonging to the same original dataset.

[0062] Step 1.3 Construct and train the TCN-VAE causal inference network model;

[0063] This invention incorporates an autoencoder (VAE) model into a Temporal Convolutional Network (TCN) for feature reconstruction, thereby enhancing the TCN's feature learning ability to construct complete causal relationships between variables, resulting in... Figure 1 The TCN-VAE causal inference network model is shown below. (Refer to...) Figure 1The improved temporal convolutional network consists of N parallel, independently operating, and structurally identical TCN channels, a TCN-VAE encoder, and a TCN-VAE decoder. The Encoder and Decoder layers in the TCN-VAE are connected after the N TCN channels to achieve depthwise separable convolution. The N TCN channels first extract the temporal features of the input data through convolution, outputting a set of temporal features corresponding to the input data. Each TCN channel is a temporal convolutional sub-network, including sequentially arranged convolutional layers, skip connection layers, and feature reconstruction layers. The hidden layers in each TCN channel are skip-connected to cascaded layers, ultimately connected in parallel. The outputs of the depthwise separable convolutional layers of the N TCN channels are connected to the TCN-VAE encoder. The TCN-VAE encoder encodes the temporal feature set output from the N TCN channels to obtain the reconstructed time-series features. The TCN-VAE decoder decodes the reconstructed time-series features from the TCN-VAE encoder output to obtain a data feature map, i.e., a causal reasoning map, which serves as the output of the entire TCN-VAE network model.

[0064] In forward convolution operations for industrial root cause reasoning, it is necessary to extract feature values ​​from a previous time period. This past time is called the receptive field of the temporal convolutional network. During dilated convolutions in the temporal convolutional network, the size of the receptive field increases exponentially to capture global features of the manufacturing process. The receptive field is calculated using the following formula:

[0065]

[0066] Where, N size L and e represent the kernel size and the depth of the convolutional layer, respectively. factor It is the expansion factor.

[0067] This invention employs a structure including fully connected layers, feature reconstruction layers, and depthwise separable convolutional layers to enhance the temporal convolutional network. The outputs of each hidden layer in the temporal convolutional network are skipped and connected in parallel to a cascaded layer. After adding an autoencoder, the temporal convolutional network consists of shallow features and deep features extracted from the shallow features, thus ensuring the reusability of features in each hidden layer and improving the extraction capability of temporal features. Furthermore, the constructed TCN-VAE encoder uses KL (Kullback Leibler) divergence to measure the difference between the reconstructed features and the original features, thereby preventing gradient vanishing during model training. Finally, a 1×1 point convolutional layer is used to merge causal relationships to obtain the feature map. The formula for the point convolutional layer is as follows:

[0068]

[0069] Where P(·) and C(·) represent point convolutional layers and concatenated layers, respectively, φ i (·) indicates a hidden layer.

[0070] Furthermore, to prevent training errors caused by the temporal convolutional network failing to find optimal weights during standard backpropagation, this invention introduces residual connections after each convolutional layer in the temporal convolutional network and uses PRelu as a non-linear activation function, thereby improving the fitting process of the temporal convolutional network model while reducing additional computational costs.

[0071] Reference Figure 1 The TCN-VAE causal inference network model of this invention has N TCN channels for feature extraction. The TCN-VAE encoder reconstructs the features extracted from these N TCN channels, enhancing the feature extraction capability of the temporal convolutional network. Finally, the TCN-VAE decoder decodes and outputs a causal inference graph, achieving depthwise separable convolution. During this process, the N manufacturing process variables operate in parallel on their respective TCN channels without interfering with each other. Furthermore, the hidden layers in each TCN network are connected to cascaded layers via skip connections. Finally, the TCN-VAE merges the reconstructed temporal features into a feature map through pointwise convolution. In the above feature map construction stage, the causal relationships between the manufacturing process variables are fully explored and quantified.

[0072] Build Figure 1 After the TCN-VAE causal inference network model is shown, it is trained and tested using the training and test sets obtained after preprocessing in step 1.2. The training dataset X is input into the TCN-VAE causal inference network model in the formats of a normal dataset (500*52) and a fault dataset (960*52). The training parameters are set according to the maximum length of the manufacturing process variable data (Length: 960), the model optimization function is Adam, the activation function is Prelude, the loss function uses KL divergence to measure the similarity between two parts, the network has 256 linear layers, and the number of iterations is 1000. Other hidden parameters, such as the data length in the linear layers and the optimal number of iterations, are determined based on the model's optimal convergence point. Once the optimal number of iterations is reached or the model test results meet the set requirements, the model training is complete, resulting in the final trained TCN-VAE causal inference network model.

[0073] Step 1.4 Obtain the causal relationship reasoning diagram;

[0074] The manufacturing variable data collected in actual production is input into the TCN-VAE causal reasoning network model in step 1.3, and the model outputs a causal relationship reasoning graph. This embodiment uses the data from fault four (IDV(4)) as an example for verification, and the resulting causal relationship inference diagram is as follows: Figure 2As shown, each node represents an operational variable. Specifically, XMV(10) and XMV(7) represent the reactor cooling water flow rate and reactor pressure, respectively. XMEAS(4) is the total feed rate (flow 4), XMEAS(7) is the reactor pressure, XMEAS(9) is the reactor temperature, XMEAS(11) is the product separator temperature, XMEAS(13) is the product separator pressure, XMEAS(18) is the stripper temperature, XMEAS(19) is the stripper flow rate, and XMV(9) is the stripper water flow valve. Figure 2 Analysis reveals that the sub-units where the reactor cooling water flow begins to occur are, in order, the process variables related to the reactor, separator, and stripper, which is consistent with the variable propagation process in the actual TE manufacturing process. Based on the root node determination, the root node scores of XMV(10) and XMV(7) are 7.89 and 6.33 respectively, indicating that the reactor cooling water flow is the root cause of this fault. Therefore, the reactor cooling water flow can be identified as the root cause of IDV(4) (i.e., fault 4).

[0075] Step 1.5 Distinguish between the direct and indirect causes of the malfunction;

[0076] Step 1.5.1 Causal relationship reasoning diagram obtained in Step 1.4 In this paper, a multi-head attention mechanism AT(·) is introduced to recalculate the attention coefficient matrix A in the causal reasoning graph, and finally, the multi-head attention mechanism is used to assign corresponding weights to each feature map.

[0077] Different time lags can generate complex causal relationships between time series. These time lags can be obtained through different subspace projections, and their similarity can be measured by scale dot products. Therefore, multi-head attention mechanisms can fully utilize the features of complex data to express the causal relationships between manufacturing process variables.

[0078] Specifically, the input feature map X θ Through the three subspace projection matrices W Q W K W V Projected onto three matrices Q, K, and V, the feature map X θ The similarity is represented by matrices Q and K, which are further used as attention weights and summed to obtain matrix V. The specific definition of the attention mechanism is as follows:

[0079]

[0080] Where Q = W Q X θ K = W K X θ V = W V X θ dk Represents the input feature map X θ The dimension is used to scale the dot product to prevent vanishing, and AT(·) represents the attention mechanism.

[0081] The attention coefficient matrix A is output to represent the degree of influence between manufacturing process variables in the prediction task. The time lag relationship between variables is used to predict the temporal causal relationship when using historical data to predict the current value.

[0082] Step 1.5.2 Based on the weight information obtained in Step 1.5.1, update the causal reasoning graph X obtained in Step 1.4 through threshold normalization and temporal feature gradient error. θ After assigning weights to each manufacturing process variable, irrelevant causal relationships in the causal reasoning graph are eliminated, further distinguishing between direct and indirect causes of failure.

[0083] A threshold normalization layer built in a network with a multi-head attention mechanism is used to extract causal relationships from the attention coefficient matrix A, where the element a in the causal relationship is... ij ∈A represents the influence of variable j on variable i, and We define g ij Used to display threshold normalization, and then to calculate a ij g varies with the number of variables ij The specific calculation formula is as follows:

[0084]

[0085] The threshold is set according to the number of variables. Attention coefficients below a threshold are considered to have low causal contribution and are therefore set to 0. Attention coefficients above the threshold are amplified to [1, N]. Significant causal relationships can be extracted through threshold normalization, and finally, the pooling layer in the multi-head attention mechanism can fuse multiple attention matrices.

[0086] Finally, causal verification is performed using temporal feature ablation based on error gradients to eliminate spurious causal relationships and confounding factors in the latent causal graph, since even if confounding factors contribute to the prediction task, they do not imply a true causal relationship. Therefore, it is necessary to verify the causal relationships in the latent causal graph. The time series are randomly sorted through random permutations to eliminate the temporal features of the variables. When the gradient of the prediction loss in the TCN-VAE causal inference network model does not change significantly, it is considered that there is no true causal relationship between the manufacturing process variables input into the TCN-VAE causal inference network model and the prediction variables.

[0087] Step 1.6 Determine the root cause of the fault;

[0088] In order to solve the fault at its root and ensure the normal operation of the industrial manufacturing process, it is necessary to determine the root cause of the fault based on the causal relationship reasoning diagram obtained after eliminating irrelevant causal relationships in step 1.5.2 and the direct and indirect causes of the fault identified in step 1.5.

[0089] In actual manufacturing processes, step 1.4 typically yields multiple causal reasoning graphs, meaning there are usually multiple causal transmission paths. Each causal reasoning graph contains a root node, making it impossible to determine the root cause from multiple root nodes. Therefore, this invention establishes a scoring rule for root nodes, considering the impact of time delays and the interaction between fault causal variables to quantitatively determine the root cause of the fault.

[0090] The score S of the root node root The calculation method is as follows:

[0091]

[0092] Where Num represents the maximum number of generation of the root node, M represents the number of iterations of the child nodes, and k n,m μ represents the causal gain of the m-th child node in the n-th iteration. n Let S represent the weights in the nth iteration. The final score S is then calculated. root The larger the value, the more important it is, and the root node corresponding to the highest score is taken as the root cause of the fault.

[0093] Step 2: Self-heal the root cause of the fault;

[0094] This invention proposes an adaptive biological immune reinforcement learning method for self-healing from the root cause of failures. First, T helper cells extract information about the failure state during the manufacturing process from the antigen patterns presented by antigen-presenting cells (APCs). Then, T helper cells regulate the interactions between effector failure variables by secreting different types of cytokines. In the reinforcement learning failure self-healing model, the T helper cell population is constructed as an intermediate layer between the failure pattern (i.e., the antigen pattern) and the activated effector cell activation pattern (e.g., ...). Figure 3 In Phase 2, when a fault occurs, antigen-presenting cells (APCs) can engulf pathogens and present various faults in different fault modes s. T helper cells recognize the fault modes s and activate effector cells to perform self-healing operations a, thereby achieving fault self-healing.

[0095] The reinforcement learning-based fault self-healing model, built upon the adaptive biological immune process of effector T cells, achieves its effect not by adjusting weights, but by adjusting the distribution of clone size. If we consider the impact of a fault on a system as an invasion of the biological immune system by an infectious agent, then when a fault is introduced into the manufacturing process, the "immune response" begins.

[0096] When the biological immune system detects a pathogen invasion, APCs can engulf the "pathogen" and present various antigens in antigenic patterns s. T helper cells recognize antigenic patterns s and trigger a self-healing response a (corresponding to the activation patterns of effector cells in biological immunity). The specific relationship between adaptive biological immunity and reinforcement learning is as follows: Figure 3 As shown, where π n (a|s) represents the multivariate random transition probability distribution from a to s, n represents the clonal parameterization system strategy of T helper cells, and R(a|s) represents the feedback reward function of effector cells that make a self-healing response a in response to antigen pattern s.

[0097] Subsequently, the APCs that engulf the pathogen begin presenting antigenic fragments of the pathogen to T helper cells. Typically, multiple antigenic fragments can represent the pathogen. We define N different types of faults and fault modes as the number of different types of antigens and the antigen (fault) mode s∈{0,1}. N ; where s i =0 and s i =1 indicates the existence of the i-th fault type, such as Figure 3 For example, the s = {0,1,1,1,1} pattern represents a fault condition where all faults except the first type are present. This fault pattern transmits information about the infection fault variable to T helper cells, and then the T helper cell population begins to secrete cytokine patterns. This fault pattern s may vary depending on the fault-induced activity of the T helper cells. Conversely, according to the cytokine pattern, different effector cell groups, such as B cells, macrophages, and T killer cells, are inactivated or activated. This process constitutes the pathogen-specific response, i.e., the response to the fault. These effector cells are different from effector T cells. Assume M represents the number of different types of effector cells, a ∈ {0,1} M This is a self-healing response, i.e., the activation mode of effector cells. Among them, a j =0 and a j =1 refers to the type of activation and deactivation of JTH effector cells, such as Figure 3 For example, a = {1, 1, 0, 1} indicates that the first, second, and fourth effector cells are activated, while the third effector cell is inactivated. Based on the effector cells s of the failure mode information, the T helper cell population can be biased towards the self-healing response a. The role of the T helper cells is represented by the random transition probability distribution π(a|s), thereby determining which self-healing response a is currently in effect. When T helper cells are exposed to the failure mode, they can achieve the antigen mode s. This invention refers to this random transition probability distribution π(a|s) as the T helper cell policy.

[0098] Ultimately, the immune system controls the failure by optimizing the feedback reward function R(a|s) to restore the manufacturing system to normal levels.

[0099] The specific steps are as follows:

[0100] Step 2.1 Construct a reinforcement learning fault self-healing model based on the adaptive biological immune process of effector T cells;

[0101] Step 2.1.1 Construct a fault self-healing model and set initial values ​​for reinforcement learning parameters:

[0102] The structure and state flow of the fault self-healing model are as follows: Figure 3 As shown. The fault self-healing model includes T helper cells, antigen-presenting cells, and activated effector cells. Using a root cause fault path dataset for causal reasoning as model input, when a fault occurs, antigen-presenting cells engulf the fault and present multiple faults as fault modes s. T helper cells recognize fault modes s and trigger a self-healing response a in effector cells. A random mapping π from fault modes s to self-healing response a is shown. n (a|s) is considered the stochastic transition probability distribution strategy of the fault self-healing model, which is parameterized by the number of T helper cell clones, n. The effectiveness feedback of the self-healing response 'a' generated by the effector cells to the fault is represented as R(a|s). The fault self-healing model learns not by adjusting weights, but by modulating the distribution n of the T helper cell clone size. Finally, the fault self-healing transition probability p(a|s) is determined by the maximum Hamming distance, thereby judging the repairability probability of the fault and deciding which fault to choose for self-healing. The final output of the fault self-healing model is the manufacturing data of the repaired process, and... Figure 5 and Figure 8 Visual presentation.

[0103] The input and reinforcement learning parameters of the constructed fault self-healing model are:

[0104] (1) There exists a set of possible fault modes, i.e., the set of fault types in the input data.

[0105] (2) A set of possible activity patterns of activated effector cells, also known as self-healing response

[0106] (3) Fault self-healing transfer probability p(a|s);

[0107] (4) Feedback reward function R(s) i ,a)∈R;

[0108] (5) The random transfer probability distribution strategy of the T helper cell population is denoted as π(a|s).

[0109] The first four parts mentioned above correspond to the fault state, self-healing operation, fault self-healing transition probability, and feedback reward function in industrial manufacturing process fault detection, respectively. The interaction between manufacturing process variables is achieved through the random transfer probability distribution strategy (also known as the cloning strategy) π(a|s) of the T-helper cell population, enabling the fault self-healing model to adaptively repair faults. The optimal cloning strategy π′ is described as the strategy that maximizes the expected cumulative feedback. t represents a time point in each manufacturing process, and 0 ≤ γ ≤ 1 represents the feedback rate, such as π′ = argmax. π J[π], where

[0110]

[0111] Among them, R π (s t ,a t () represents the feedback reward function under the optimal cloning strategy, with additional conditions. Let represent the control cost of a manufacturing process variable deviating from its normal threshold behavior by π to π0, where β∈(0,∞) is the scaling parameter for the control cost. t and a t Let represent the failure mode and the activation (self-healing) mode at time t during the entire manufacturing process, respectively. Considering the functional form of controlling consumption, this formula can also be expressed as Entropy Regularized Reinforcement Learning (ERRL). For simplicity, we assume a uniform normal threshold π0, then the optimal cloning strategy can be expressed as follows:

[0112]

[0113] The optimization function Q is defined as Q = max π The function Q(s,a) is defined as follows:

[0114]

[0115] Where s0 and a0 are s t and a t The initial value.

[0116] Step 2.1.2 Optimize fault self-healing based on T helper cells to obtain the activity h of υ-type faults. ω (s);

[0117] Each T helper cell is characterized by its T cell receptor (TCR) and the types of cytokines it secretes. Each T helper cell clone can be characterized by the interaction between the T cell receptor (TCR) and effector cells.

[0118] In this invention, V is defined as the number of clones of different T helper cells, and nυ This represents the size of the υth generation T helper cell clone. Inspired by the affinity of the T cell receptor (TCR) for antigens and the way the antigen itself is expressed, each T helper cell clone interacts with its corresponding fault variable at varying intensities.

[0119] Since each T helper cell clone has a unique T cell receptor (TCR), the self-healing strength between the v-th T helper cell clone and the i-th fault is... All T helper clones in the vth T helper cell clone are the same.

[0120] When the fault mode is s, each type υ T helper cell should be stimulated by the i-th fault, and the total stimulation received by each type υ T helper cell from the i-th fault becomes In reality, the occurrence of a malfunction in a T helper clone is probabilistic; only a small fraction of T helper clones within each T helper clone will experience malfunction. Therefore, It should be interpreted as the average expected failure level per T helper clone in an industrial manufacturing system.

[0121] When a υ-type fault occurs, its activity h ω (s) is considered to be significantly dependent on the stimulus intensity of the s-type failure mode, based on the average acceptable expected failure level. The activity dependent on S-mode failure stimulation can be calculated, specifically defined as follows:

[0122]

[0123] Where σ represents the Sigmoid function, whose activity h is determined when the υ-type fault occurs or disappears completely. ω They become 1 or 0 respectively.

[0124] Step 2.1.3: Modulate the fault self-healing transfer probability for different average stimulus intensities;

[0125] The fault activity h calculated according to step 2.1.2 ω T helper cells release cytokines, an activity that in turn stimulates the activity of effector cells. Assuming all T helper cell types v release the same type of cytokines, the average stimulatory intensity of the cytokines released by type v T helper cells on type j effector cells is expressed as: Among them, h v It is the activity of the kth T helper clone, u jv ∈R represents the stimulus intensity, and β∈(0,∞) is a global scaling parameter that can be identified using β introduced in reinforcement learning above.

[0126] The j-th type of activated effector cells, upon receiving the overall fault stimulus from all T helper cells, give the following response: v v n v h v u jv In response to this fault stimulus, type j activated effector cells are activated by the s-mode fault (a) j =1) or inactivation (a j The probability of (=0) is modulated according to the fault self-healing transfer probability, which is expressed as follows:

[0127]

[0128] Cytokines are the main active agents for T helper cell-induced effector cell activity. Notably, under the aforementioned failure-to-heal metastasis probability, activated effector cells possess the autonomous capacity to be activated or inactivated, a capability influenced by biases generated by signals from T helper cells.

[0129] Although the autonomous activity of activated effector cells is directly driven by faults, to simplify the immune system's response process, this invention assumes that effector cell activity is regulated solely by T helper cells and focuses primarily on the role of T helper cells. Therefore, the T helper cell population transforms the fault type it receives into a self-healing response 'a' of activated effector cells. Let π be the random transition probability of the T helper cell population receiving a certain fault type, transforming it into an activation mode, and thus initiating a self-healing response 'a'. n The specific definition is as follows:

[0130]

[0131]

[0132] Where, the random transition probability π n (a|s) is the strategy of the immune system executed by the T helper cell population, whose role is to update the strategy over time by adjusting the clonal size distribution n, thereby increasing the random transition probability π. n To get closer to the optimal cloning strategy π′, and thus obtain a greater reward. This represents the global negative energy scaling parameter. In the above representation, the random transition probability π n (a|s) is likely to select the self-healing response a of an effector cell, a strategy that selects self-healing response a with a larger response than other cells. Value. If If the global negative energy scaling parameter value of the self-healing response a is selected when fault mode s is represented, then the probability π of random transfer implemented by the T helper cell population is... n (a|s) can be interpreted as having a larger choice. The probability of a self-healing response is higher than that of other strategies.

[0133] Step 2.1.4 Determine the feedback reward function R(s) i ,a)∈R;

[0134] To simplify the immunization process, this invention performs homogeneous modeling optimization of the parameters of T helper cell clones, focusing on the general properties of the model. Depending on the objective, specific subtypes and structures can be included as extensions. Therefore, we present M different faults and z different fault modes {s}. 1 ,s 2 ,…,s i ,...,s z} represents the states of no fault and fault occurrence, where s 1 For the fault mode where no fault occurred, s i This represents the i-th generation failure mode. The invention also assumes the existence of K different types of effector cells and Z associated possible activity modes (self-healing responses) {a} 1 ,a 2 ,…a Z Each possible activity pattern represents the most effective self-healing mode of the effector cell in response to the corresponding fault. This invention incorporates more specific dynamics and similarities of faults into the reinforcement learning fault self-healing model, for each fault pattern s... i , i∈{1,…,z}, feedback reward function R(s) i a) as follows:

[0135] R(s i ,a)=RW-ham(a i a)

[0136] Among them, a i For fault mode s i The most effective mode of activity, ham(a i ,a) is a binary vector a i The Hamming distance between and a. This form of the feedback reward function indicates that when the self-healing response a is related to the failure mode s... i The most effective active self-healing mode a i When matched, the immune system receives the highest reward (RW). If the self-healing response deviates from the most effective active self-healing pattern... i The immune system will lose its reward due to deviation (a i ,a). Because the reward (a) i a) is the maximum Hamming distance between a pair of carriers of length RW, so the feedback reward function R(s) i a) is positive for any pair of fault and active modes.

[0137] Step 2.1.5 Select the self-healing transfer probability p(a|s);

[0138] The specific form of the self-healing transfer probability P(s|s,a) selected in this invention is as follows:

[0139]

[0140] The dynamic process of the self-healing transfer probability p(a|s) is as follows: Figure 4 When a fault is detected in a normal state, the reinforcement learning self-healing model will adjust the self-healing transition probability P(s) according to the fault condition. j |s j a) Repair the fault. Where q is the probability of a fault randomly selected from Z-1 faults. If the fault occurs, the probability that the fault is repaired is... This means that if the reward R(s) i If a) is the largest, then the probability of the fault being repaired is 1. On the other hand, if the reward R(s) is the largest... i a) If the value of the fault is less than or equal to the threshold R0, then the probability of the fault being repaired is 0, meaning the fault cannot be repaired. With the reward R(s)... i a) As R0 increases, the chance of recovery also increases. When R0 is small, due to the inherent ability of effector cells, the fault can be cleared from T helper cells with a certain probability, even without effective control. If R0 approaches the maximum reward, reinforcement learning is often hindered by being trapped in one of the fault states. This indicates that the innate ability to recover from faults is a necessary condition for adaptive learning. This invention considers more complex situations by expanding the space of fault modes s. For example, by assigning several states of fault modes s to each fault to represent the stages of fault occurrence, the transitions between these stages depend on the action of the immune system. Such a model can be effectively used to analyze more complex fault types, such as slow-drift faults, etc.

[0141] Step 2.2 Input the root cause of the failure and the set of related failure manufacturing process variables obtained in Step 1.6 into the reinforcement learning failure self-healing model established in Step 2.1 for self-healing repair. The repair result is as follows: Figure 5 As shown. Figure 5 The fault type repaired was a random fault. From the results, it can be seen that the self-healing method proposed in this invention can be applied to faults in the T... 2 Both the SPE and SPE metrics are well adapted to changes in random failure types and provide self-healing control for each failure exceeding the threshold, ensuring the safe and reliable operation of the manufacturing process.

[0142] Performance verification:

[0143] The present invention will now analyze the stability control and repair accuracy of the constructed reinforcement learning fault self-healing model.

[0144] I. Stability Control Analysis of Fault Self-Healing in Reinforcement Learning

[0145] By focusing on the feedback reward function R(s) of T helper cells during the repair process i The influence of the input to the reinforcement learning fault self-healing model on the output is analyzed to assess the accuracy, robustness, and reliability of the overall industrial manufacturing process after fault repair.

[0146] This invention focuses on the feedback reward function of T helper cells during the repair process, and statistically displays the numerical fluctuations during the repair process using curve data from 100 independent learning iterations. Figure 6 As shown in the diagram. Yellow represents the average reward function after 100 training iterations, while blue and gray represent the range between the 25th and 75th rewards, respectively. Figure 6 Analysis shows that the reward function curve of the T helper cell converges and stabilizes during 100 training iterations. Therefore, the reinforcement learning fault self-healing model proposed in this invention is reliable in the fault repair process. The average reward function is monotonically increasing in each training round and eventually stabilizes, indicating that by updating the clone size distribution, the fault self-healing reward function can obtain a larger reward during the dynamic adjustment process over time.

[0147] Furthermore, in intelligent manufacturing processes, there are many types of faults, and fault variables influence each other, resulting in significant input uncertainty and consequently, significant output uncertainty. Therefore, in addition to stability analysis, this invention also needs to analyze the robustness of the reinforcement learning fault self-healing model. Here, we use global sensitivity to characterize it. To quantify the variance of each input variable and further analyze the interaction between various manufacturing process variables, this invention uses the SOBO method to perform global sensitivity analysis on the inputs and outputs of the reinforcement learning fault self-healing model for different fault types. Based on variance decomposition, this invention studies the quantitative measurement of the contribution of inputs to output variance, and uses the same principle as classical analysis of variance in factor design to express the output variance as follows:

[0148]

[0149] Where V represents the total variance of the output of the reinforcement learning fault self-healing model of this invention, V i V represents the parameters of the first-order contribution ith reinforcement learning fault self-healing model. ij Let ith and jth represent the interaction between model variables in reinforcement learning fault healing. Based on this decomposition, the global first-order sensitivity exponential equation and higher-order sensitivity exponential equations of the reinforcement learning fault healing model are given:

[0150]

[0151]

[0152]

[0153] Among them, S i The first-order sensitivity index is used to quantitatively measure the input parameter X. i The contribution of the main effects to the output variance; S ij S represents the second-order sensitivity index, which can quantitatively measure the contribution of the interaction between model ith and jth to the output variance; ijk Let n represent the third-order sensitivity index, and so on for the other higher-order sensitivity indices. The total number of sensitivity indices constructed in this way, from order 1 to n, is (2^n - n^n). n -1). When the number of input variables is too large, the sensitivity index will explode. Therefore, a total sensitivity index is introduced. To represent the manufacturing process variable X i The overall sensitivity to the variance Y caused by the total sensitivity. Overall sensitivity index. It can be defined as being composed of X i The sum of contributions from the interaction of other variables is given by the following formula:

[0154]

[0155] Among them, S σ Indicates each order S i , σ i This represents the sum from i to j in each order. The sensitivity analysis results of this invention are as follows: Figure 7 When the SOBO sensitivity index is closer to 1, the variable parameters in the reinforcement learning fault self-healing model become more important. Figure 7 As can be seen from the data, the SObol sensitivity index is mostly distributed between 0 and 1, meaning that the variance of the influence of different input values ​​on the output is small. Therefore, the fundamental fault self-healing model proposed in this invention has good stability.

[0156] II. Accuracy Analysis of Reinforcement Learning Fault Self-Healing Model for Fault Self-Healing Repair

[0157] This invention analyzes the self-healing results from the perspectives of causal relationship reasoning diagrams, self-healing repair results, and three-dimensional visualization processes, as detailed in the following figures. Figure 5 Root cause of the fault self-healing result diagram and Figure 8 A 3D visualization of the process failure self-healing results. A biological immune reinforcement learning model was used to regulate the self-healing of the root cause of the failure, and the results are as follows. Figure 5 The results show that the self-healing method proposed in this invention can be derived from T. 2Both the SPE and S2000 metrics can adapt well to changes in random failure types and perform self-healing control for each failure exceeding the threshold, ensuring the safe and reliable operation of the intelligent manufacturing process. Through the analysis of... Figure 8 Analysis of the process repair results for randomly changing fault types shows that the model can promptly repair 12 relevant variables on the propagation path of random faults, and can achieve complete fault repair from the fault occurrence location to the root node location.

[0158] In summary, this invention can enhance the feature learning ability of networks and analyze direct and indirect causes to construct complete variable causal relationships, and achieve self-healing from the root of the fault, solving the fault at its source and improving manufacturing efficiency. It has high application value and practical significance in the process of intelligent manufacturing.

Claims

1. A self-healing method for faults in industrial manufacturing processes, characterized in that, Includes the following steps: Step 1: Collect manufacturing variable data related to the production process through simulation. After introducing known fault types for labeling, divide the data into normal production dataset and faulty production dataset, and use them as training set and test set respectively. Step 2: Preprocess the manufacturing variable data in the training and test sets by removing non-numeric characters and standardizing them; Step 3: Construct the TCN-VAE causal inference network model and set its initial parameters. Train the network model using the data in the training set after preprocessing in Step 2. During training, update and iterate the network parameter settings through the loss function and optimization function, and test the model using the test set until the set requirements are met, at which point the model training terminates. The initial parameters include input format, receptive field, convolutional kernel size, convolutional layer depth, maximum length of process variable data, model optimization function, activation function, number of iterations, loss function, and network layer structure. Step 4: Input the manufacturing variable data collected in actual production into the TCN-VAE causal reasoning network model in Step 3. The TCN-VAE causal reasoning network model outputs a causal relationship reasoning graph. Step 5: In the causal relationship inference graph obtained in step 4, a multi-head attention mechanism is introduced to calculate and update the weights of each feature graph, and irrelevant causal relationships in the causal relationship inference graph are eliminated by time feature gradient error, so as to further distinguish the direct and indirect causes of the failure. Step 6: Based on the causal reasoning graph obtained after eliminating irrelevant causal relationships in Step 5, and the distinction between direct and indirect causes in Step 5, establish and quantitatively determine the root cause of the fault by considering the impact of time delay and causal relationship gain according to the scoring rules of the root node. Step 7: Use adaptive biological immune reinforcement learning to self-heal the root cause of the failure obtained in Step 5. The TCN-VAE causal inference network model described in step 3 above includes N parallel, independently operating, and structurally identical TCN channels, as well as a TCN-VAE encoder and a TCN-VAE decoder; the outputs of the N TCN channels are connected to the input of the TCN-VAE encoder, and the output of the TCN-VAE encoder is connected to the input of the TCN-VAE decoder; the N TCN channels are used for convolutional extraction of temporal features; the TCN-VAE encoder is used to reconstruct time series features; The TCN-VAE decoder is used to decode the time-series features reconstructed by the TCN-VAE encoder and output a causal inference graph. The root node scoring rule established in step 6 above is as follows: calculate the score of each root node, and take the root node with the highest score as the root cause of the fault. The formula for calculating the fraction of the root node is: in, This represents the maximum number of generated nodes for the root node. This represents the iteration number of the child node. Indicates the first In the nth iteration Causal gain of each child node Indicates the first The weights for the next iteration. The score of the root node.

2. The self-healing method for industrial manufacturing process faults according to claim 1, characterized in that: The TCN-VAE encoder uses KL divergence to measure the difference between the reconstructed features and the original features when reconstructing time series features.

3. The self-healing method for industrial manufacturing process faults according to any one of claims 1-2, characterized in that: In step 5, the method for introducing a multi-head attention mechanism to calculate each feature map in the causal reasoning graph is as follows: The formula for establishing an attention mechanism from the input feature map through three subspaces is as follows: in, , Represents the input feature map The dimension is used to scale the dot product to prevent vanishing. This represents the attention mechanism. Let Q be the subspace projection matrix, and let K be the matrix representing the similarity of the feature maps. V is the matrix obtained by weighting Q and K as attention weights. This is the attention coefficient matrix, representing the degree of influence between manufacturing process variables.

4. The self-healing method for industrial manufacturing process faults according to claim 1, characterized in that: Step 7 specifically includes: Step 7.1 Construct a reinforcement learning fault self-healing model based on the effector T cell immune process, set initial values ​​for reinforcement learning parameters, and perform self-learning iterations on the reinforcement learning parameters; the fault self-healing model includes T helper cells, antigen-presenting cells, and activated effector cells; when a fault occurs, antigen-presenting cells are used to engulf pathogens and present multiple faults in different fault modes of the manufacturing process variables; T helper cells are used to extract fault state information in the manufacturing process from the fault modes presented by antigen-presenting cells, and enable the activated effector cells to perform a self-healing operation response to the fault state information, clearing the fault from the T helper cells with a certain probability; the self-learning is achieved by modulating the distribution of T helper cell clone size; Step 7.2 Input the root cause of the fault obtained in Step 6 and the set of fault manufacturing process variables related to the root cause collected during the production process into the reinforcement learning fault self-healing model constructed in Step 7.1 for self-healing repair.

5. The self-healing method for industrial manufacturing process faults according to claim 4, characterized in that: The reinforcement learning parameters of the fault self-healing model include: The self-healing response generated by activated effector cells; The fault self-healing transfer probability is used to modulate the probability of activated effector cells being activated or inactivated by various fault modes. When a fault is detected, the activated effector cells repair the fault according to the fault self-healing transfer probability. The feedback reward function is used to characterize the effectiveness of the self-healing response generated by activated effector cells in responding to the fault. When the self-healing response generated by activated effector cells deviates from the most effective active self-healing pattern, the immune system loses the reward; when the self-healing response generated by activated effector cells matches the most effective active self-healing pattern, the immune system receives the highest reward. The random transfer probability distribution strategy of the T helper cell population is used to realize the interaction between manufacturing process variables, so that the fault self-healing model can make adaptive repairs to faults.

Images