Deep fusion network production line fault prediction method based on deep learning

Abstract

The invention discloses a deep learning-based deep fusion network production line fault prediction method, which comprises the following steps of: firstly, acquiring SCADA (Supervisory Control And Data Acquisition) system or sensor data of an automatic production line, and carrying out data preprocessing, feature extraction and fault prediction label construction; then a deep fusion network 1D-CNN-LSTM-Attention-DCE model combining a one-dimensional convolutional network, a long short-term memory neural network, a fusion attention mechanism and a dynamic cross entropy loss function DCE is established, the preprocessed data is input into the constructed model for training, and finally the trained model is used for fault prediction to obtain a fault prediction result. According to the method, a 1D-CNN-LSTM-Attention-DCE fault prediction model is provided, local features are extracted from time sequence data by using the 1D-CNN, then a long-time dependency relationship is captured through LSTM, and finally weight distribution of key features is enhanced through an attention mechanism, so that the processing capability of the fault prediction model on complex time sequence data is comprehensively improved, and the fault prediction efficiency is improved. And the fault of the production line can be accurately predicted within a certain time length range.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault prediction in industrial production, and specifically relates to a deep fusion network production line fault prediction method based on deep learning. Background Art

[0002] With the rapid development of the global manufacturing industry, the automation and intelligence of industrial production lines are constantly improving. However, the failure of production equipment is still one of the main bottlenecks restricting production efficiency and product quality. Existing equipment maintenance strategies mainly include reactive maintenance (repair after the failure occurs) and preventive maintenance (regular inspection based on time or usage cycle), but these methods have certain limitations, especially when dealing with complex and changing production environments, it is often difficult to achieve efficient and economical maintenance. In recent years, predictive maintenance (PDM) technology has gradually become a more advanced maintenance strategy. It predicts the possibility of failure by analyzing the historical data and real-time operation data of the equipment, so as to take preventive measures before the failure occurs and avoid unplanned downtime of the production line.

[0003] In this context, deep learning technology, as an important branch of artificial intelligence, has become a key tool in fault prediction technology due to its excellent performance in processing complex nonlinear data and capturing deep-level features. Deep learning technology, especially models based on architectures such as convolutional neural networks (CNN) and long short-term memory networks (LSTM), can automatically extract effective features from massive amounts of sensor data and accurately predict the operating status and potential faults of equipment by learning from historical operating data.

[0004] Equipment in industrial production lines often generate a large amount of heterogeneous and time-series operating data, which includes both physical parameters of the equipment, such as temperature, vibration, pressure, etc., as well as interaction information between devices. Existing machine learning methods have limited performance in complex industrial environments due to their heavy reliance on feature extraction and their inability to effectively process high-dimensional, nonlinear, and time-series data. Deep learning technology can automatically extract high-dimensional features from data through an end-to-end learning process, and use these features to model and predict the health status of equipment, greatly improving the accuracy and real-time performance of fault prediction.

[0005] Predictive maintenance has gradually transformed from existing experience-based and simple rule-based technologies to intelligent solutions based on advanced technologies such as deep learning. Through the cloud-edge collaborative computing architecture, industrial enterprises can deploy complex deep learning models in the cloud, use the powerful computing power of the cloud to train and optimize large amounts of data, and perform real-time data analysis and fault prediction on edge devices. This architecture not only improves the real-time nature of predictions, but also solves the latency and security issues during data transmission. However, existing models often have difficulty processing high-dimensional sensor data and solving the imbalanced distribution of fault samples, which greatly hinders the performance of the model. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a deep fusion network production line fault prediction method based on deep learning, which solves the problem that the existing model is difficult to process high-dimensional sensor data and solves the imbalanced distribution of fault samples.

[0007] The technical solution adopted by the present invention is: a deep fusion network production line fault prediction method based on deep learning, the specific steps are as follows:

[0008] S1, collect SCADA system or sensor data of automatic production line, perform data preprocessing, feature extraction and fault prediction label construction;

[0009] First, the relevant state quantities of the automatic production line operation are collected through sensors or SCADA systems, and the data is preprocessed, namely sliding window processing.

[0010] Among them, the relevant state quantities of the production line operation are collected through sensors or SCADA systems, namely SCADA system or sensor data, including: temperature, current, voltage, vibration, pressure, humidity, gas and liquid flow. The statistical information obtained by sliding window processing data includes: mean, median, sum, standard deviation, maximum and minimum.

[0011] Then, feature extraction is performed on the data processed by the sliding window, including: automatic production line fault logs, maintenance logs, component replacement records, and robot metadata.

[0012] Finally, combined with the extracted features, fault prediction labels are constructed for the preprocessed data, and equipment operation data of multiple automatic production lines that perform the same work tasks are collected.

[0013] The process of constructing fault labels for preprocessed data is as follows: a time window is drawn for the collected data according to the fault time point, and the production line-related component data falling into the time window corresponding to the fault point is marked as faulty, and the rest of the data is marked as normal.

[0014] S2. Based on step S1, a deep fusion network 1D-CNN-LSTM-Attention-DCE model combining a one-dimensional convolutional network, a long short-term memory neural network, a fusion attention mechanism, and a dynamic cross entropy loss function DCE is established, i.e., a fault prediction model;

[0015] First, a one-dimensional convolutional neural network is established, which can slide the convolution kernel on the one-dimensional space of the input data to extract features; secondly, a long short-term memory network is established to capture the long-term dependencies between data; then, an attention mechanism is added to analyze the encoded features to improve the quality and representativeness of the encoded features in the network; finally, an imbalance factor is introduced in the dynamic cross entropy function to update the weights in real time according to the classification results of each epoch during model training.

[0016] S3, inputting the data pre-processed in step S1 into the fault prediction model constructed in step S2 for training;

[0017] To train the 1D-CNN-LSTM-Attention-DCE model, first input the processed sample data into the 1D-CNN module to extract local features, then input the intermediate variables processed by the swish activation function into the LSTM module to extract temporal features, and finally the data flows into the fusion attention module.

[0018] S4. Perform fault prediction using the fault prediction model trained in step S3 to obtain a fault prediction result.

[0019] Furthermore, the step S2 is specifically as follows:

[0020] S21. Establish a one-dimensional convolutional neural network;

[0021] Set (x1, x2, …, x n ) represents the time series input feature value, (o1,o2,…,o n ) represents the high-level feature value obtained after the convolution operation.

[0022] Among them, n represents the sequence length; the convolution kernel and input data of the one-dimensional convolutional neural network 1D-CNN are both one-dimensional, including: one-dimensional convolution layer and pooling layer.

[0023] The convolution layer slides a fixed-size window over the input sequence. Each convolution operation outputs the dot product of the elements in the window and the convolution kernel elements, and its calculation expression is as follows:

[0024]

[0025] Among them, x represents the input sequence, w represents the convolution kernel, t represents the time step, a represents the index variable, that is, the position of the sliding convolution window in the input signal x and the convolution kernel w, k represents the convolution kernel size, and y(t) represents the convolution output.

[0026] After the data is convolved, it enters the pooling layer and performs a pooling operation on the data. The calculation expression of the pooling operation is as follows:

[0027] y(t)1=max(x(t),x(t+1),…,x(t+p-1))

[0028] Among them, p represents the size of the pooling window and y(t)1 represents the pooling output.

[0029] S22, based on step S21, establishing a long short-term memory network;

[0030] After the running data of the production line components are processed by the one-dimensional convolutional neural network 1D-CNN, the extracted feature data will enter the long short-term memory network LSTM, and the data will be gradually input into the LSTM network in the time series. For each time step, the input data x t The hidden state h of the previous time step t-1 The forget gate determines the forget information based on the current input and the previous hidden state, and calculates the forget gate output f through a Sigmoid activation function. t , whose value is between 0 and 1, represents the proportion of information that should be retained or forgotten. t The calculation expression is as follows:

[0031] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0032] Among them, W f and b f represents the weight and bias of the forget gate, σ represents the sigmoid activation function, and f t Used to adjust the previous unit state C t-1 , delete unnecessary information.

[0033] Set the input gate, first use the Sigmoid function to calculate the value to be updated, and then use the Tanh function to generate a new candidate vector Input gate output i t With candidate vector The calculation expression is as follows:

[0034] i t =σ(W i·[h t-1 ,x t ]+b i )

[0035]

[0036] Among them, W i represents the input gate weight matrix, b i represents the input gate bias, W c represents the weight matrix, b c A bias term representing the candidate state.

[0037] Input gate output i t Controls the new candidate vector How to add unit status. Updated unit status C t Determined by the calculation results of the forget gate output and the input gate, C t The long-term information required by the network is retained, and its calculation expression is as follows:

[0038]

[0039] Set the output gate to determine the output h of the LSTM unit t , through the current cell state C t With input o t Combined, calculated by Tanh and Sigmoid functions. The output gate controls the output information of the final LSTM unit. t With o t The calculation expression is as follows:

[0040] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0041] h t =o t *tanh(C t )

[0042] Among them, W o represents the output gate weight matrix, b o Represents the output gate bias.

[0043] S23, based on step S22, adding an attention mechanism to analyze the encoding features;

[0044] After the data is processed by 1D-CNN-LSTM, a fusion attention mechanism is added to assign different weights to different feature maps.

[0045] The fusion attention mechanism is based on the multi-head attention MHA mechanism and the external attention mechanism EA. The attention mechanism is added to analyze the encoding features. The specific operation process is as follows:

[0046] 1) Establish a multi-head attention mechanism MHA;

[0047] The basic form of MHA is as follows:

[0048]

[0049] Where Q represents the query matrix, K represents the key matrix, and d k represents the dimension of query and key, V represents the value matrix, Attention(Q,K,V) represents the output representation obtained by finding matching features from the key matrix K based on the query matrix Q and performing weighted summation on the value matrix V, and T represents the transpose of the matrix.

[0050] Different aspects and subspace features of the input are captured through multiple independent self-attention heads, and each attention head performs self-attention calculations. The calculation expression is as follows:

[0051] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O

[0052] Among them, MultiHead(Q,K,V) represents the output representation of multiple attention heads, W O Represents a linear transformation, merging the inputs for different heads; each head i Represents an independent attention function, and the calculation expression is as follows:

[0053] head i =Attention(QW i Q ,KW i K ,VW i V )

[0054] Among them, W i Q ,W i K and W i V The projection matrix representing the query, key, and value maps the input to different representation spaces. The subscript h of head represents a hyperparameter whose range is determined by the model and has no fixed limit. The output of each attention head is linearly transformed and concatenated together to obtain the output of the multi-head attention mechanism.

[0055] 2) Establish an external attention mechanism EA;

[0056] The fusion attention mechanism sends the features processed by the SA module to the EA module. First, the shared matrix M is constructed k and M v To focus on the potential correlation between different samples. Then use M k For feature P∈R m×n The attention map between the dataset and the shared matrix is ​​processed, and then the softmax function is used to normalize the attention map to obtain the scoring matrix.

[0057] Where R represents a real number matrix, m represents the number of feature dimensions, and n represents the sequence length. The scoring matrix is ​​composed of M v Fine-tune and output the final features. The EA module uses M k and M v Implicitly process the data set. The process expression of the EA mechanism is as follows:

[0058]

[0059] A=M v (s)

[0060] Among them, P j represents the j-th feature output by the SA module, s represents the score matrix, N represents the input sequence length, and A represents the attention matrix.

[0061] S24, add dynamic cross entropy loss function DCE;

[0062] The process expression of the DCE mechanism is as follows:

[0063]

[0064] Among them, y pre Represents the predicted label category, y true represents the actual label category, represents the imbalance factor at the tth training epoch, which is calculated based on the fault prediction result of the t-1th epoch and the initial imbalance factor. represents the initial imbalance factor, which represents the ratio of the number of samples of each fault category to the maximum category, n c represents the total number of samples, and c represents the total number of categories.

[0065] The parameters involved in the DCE mechanism include: training cycle T; batch size B; classifier parameter θ; the working process is as follows:

[0066] 1) Input data set A0 = [A1, A2, ..., A n], the attention matrix A is split into small batches A of size B B , label y = [y1,y2,…,y n ].

[0067] in, represents the jth feature after feature transformation, y j Indicates a unique encoding format.

[0068] 2) Count the number of samples of each fault type in the training data set, and calculate the imbalance factor for each fault category based on the inverse ratio of the number of samples

[0069] in, It represents the relative rarity of the i-th type of fault. The rarer the type, the larger the factor.

[0070] 3) Through A B Compute the loss and gradients:

[0071]

[0072] in, Represents the derivative of the loss function with respect to the model.

[0073] 4) Use Adam optimizer to update the classifier parameters θ(θ,β1,β2,μ,m0,v0)

[0074]

[0075] Among them, β1, β2 represent the decay parameters used to control the momentum and gradient square estimation, μ represents the initial value of the learning rate, m0, v0 represent the initial values ​​of the first-order matrix and the second-order matrix estimation, m t , v t represents the estimated values ​​of momentum and squared gradient at the tth training iteration, represents the bias-corrected momentum and squared gradient estimates.

[0076] 5) Training is performed in multiple epochs, and model parameters are updated cyclically. At the beginning of the tth training epoch, the imbalance factor is updated using the prediction results of the previous epoch combined with the initial imbalance factor. For each fault category j, check the number of samples that are misclassified in the current epoch and obtain the labeling matrix M t [j],M t [j] indicates whether the j-th fault is misclassified in the current epoch.

[0077] The specific rules for updating the imbalance factor are as follows:

[0078] If Mt [j] = 0, that is, no misclassification, then the imbalance factor remains unchanged, a t [j] = a t-1 [j];

[0079] If M t [j]≠0, that is, misclassification, then adjust the imbalance factor, a t [j]=M t [j]×a0[j], that is, enlarging or reducing the initial imbalance factor a0[j] based on the degree of misclassification, and dynamically adjusting the penalty intensity of the loss function on this category.

[0080] Among them, a t [j] represents the imbalance factor of the jth category in the tth round of training, which is dynamically updated as the training progresses, and a0[j] represents the initial imbalance factor of the jth category.

[0081] Beneficial effects of the present invention: The method of the present invention first collects the SCADA system or sensor data of the automatic production line, performs data preprocessing, feature extraction and fault prediction label construction, and then establishes a deep fusion network 1D-CNN-LSTM-Attention-DCE model that combines a one-dimensional convolutional network, a long short-term memory neural network, a fusion attention mechanism and a dynamic cross entropy loss function DCE, inputs the preprocessed data into the constructed model for training, and finally uses the trained model to perform fault prediction to obtain a fault prediction result. The method of the present invention proposes a 1D-CNN-LSTM-Attention-DCE fault prediction model, which uses 1D-CNN to extract local features from time series data, and then uses LSTM to capture long-term dependencies, and finally strengthens the weight distribution of key features through the attention mechanism, which comprehensively improves the fault prediction model's ability to handle complex time series data, and can accurately predict faults within a certain time length range for the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 This is a flow chart of a deep fusion network production line fault prediction method based on deep learning of the present invention.

[0083] Figure 2 Schematic diagram of a sliding window for performing sliding window processing on data in an embodiment of the present invention.

[0084] Figure 3 Schematic diagram of a one-dimensional convolution operation in an embodiment of the present invention.

[0085] Figure 4 Schematic diagram of one-dimensional convolution and pooling operations in an embodiment of the present invention.

[0086] Figure 5This is a schematic diagram of the structure principle of the multi-head attention mechanism in an embodiment of the present invention.

[0087] Figure 6 Schematic diagram of the structure principle of the external attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] The method of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0089] like Figure 1 As shown, a flow chart of a deep fusion network production line fault prediction method based on deep learning of the present invention, the specific steps are as follows:

[0090] S1, collect SCADA system or sensor data of automatic production line, perform data preprocessing, feature extraction and fault prediction label construction;

[0091] First, the relevant state quantities of the automatic production line operation are collected through sensors or SCADA systems (Supervisory Control And Data Acquisition), and the data is preprocessed, that is, sliding window processing. This embodiment introduces sliding window processing technology in the data preprocessing stage, and labels the data in combination with production line fault logs and maintenance logs. This method not only improves the timeliness of the data, but also ensures the accuracy of the label construction, so that the model can better capture fault characteristics during the training process.

[0092] Among them, the relevant state quantities of the production line operation are collected through sensors or SCADA systems, namely SCADA system or sensor data, including: temperature, current, voltage, vibration, pressure, humidity, gas and liquid flow. The statistical information obtained by sliding window processing data includes: mean, median, sum, standard deviation, maximum and minimum.

[0093] Then, feature extraction is performed on the data processed by the sliding window, including: automatic production line fault logs, maintenance logs, component replacement records, and robot metadata. In this embodiment, the sliding window diagram for sliding window processing of data is as follows: Figure 2 As shown in the figure, t1, t2, …, t i ,t i1 ,…,t n represents the time step, W1,W2,…,W i ,…,W n-T represents the sliding window, and T represents the window length.

[0094] Finally, combined with the extracted features, fault prediction labels are constructed for the preprocessed data, and equipment operation data of multiple automatic production lines that perform the same work tasks are collected.

[0095] The process of constructing fault labels for preprocessed data is as follows: a time window is drawn for the collected data according to the fault time point, and the production line-related component data falling into the time window corresponding to the fault point is marked as faulty, and the rest of the data is marked as normal.

[0096] S2. Based on step S1, a deep fusion network 1D-CNN-LSTM-Attention-DCE model combining a one-dimensional convolutional network, a long short-term memory neural network, a fusion attention mechanism, and a dynamic cross entropy loss function DCE (Dynamic Cross Entropy loss) is established, i.e., a fault prediction model;

[0097] First, a one-dimensional convolutional neural network is established, which can slide the convolution kernel on the one-dimensional space of the input data to extract features; secondly, a long short-term memory network is established to capture the long-term dependencies between data; then, an attention mechanism is added to analyze the encoded features to improve the quality and representativeness of the encoded features in the network; finally, an imbalance factor is introduced in the dynamic cross entropy function to update the weights in real time according to the classification results of each epoch during model training.

[0098] S3, inputting the data pre-processed in step S1 into the fault prediction model constructed in step S2 for training;

[0099] 1D-CNN-LSTM-Attention-DCE model training is performed. In this embodiment, the processed sample data (64, 30) is first input into the 1D-CNN module to extract local features, and then the intermediate variables processed by the swish activation function are input into the LSTM module to extract time series features. Finally, the data flows into the fusion attention module.

[0100] S4. Perform fault prediction using the fault prediction model trained in step S3 to obtain a fault prediction result.

[0101] In this embodiment, step S2 is specifically as follows:

[0102] S21. Establish a one-dimensional convolutional neural network;

[0103] The operation process of one-dimensional convolution is as follows Figure 3 As shown, set (x1, x2, …, x n ) represents the time series input feature value, (o1,o2,…,o n ) represents the high-level feature value obtained after the convolution operation.

[0104] Where n represents the sequence length; the convolution kernel and input data of the one-dimensional convolutional neural network 1D-CNN are both one-dimensional, including: one-dimensional convolution layer and pooling layer. Figure 4 As shown in the figure, [x0,x1,x2,…] represents the input sequence and [w0,w1,w2,…] represents the convolution kernel.

[0105] The convolution layer slides a fixed-size window over the input sequence. Each convolution operation outputs the dot product of the elements in the window and the convolution kernel elements, and its calculation expression is as follows:

[0106]

[0107] Among them, x represents the input sequence, w represents the convolution kernel, t represents the time step, a represents the index variable, that is, the position of the sliding convolution window in the input signal x and the convolution kernel w, k represents the convolution kernel size, and y(t) represents the convolution output.

[0108] After the data is convolved, it enters the pooling layer. The pooling layer is located after the convolution layer. Its purpose is to reduce the spatial size of the data, thereby reducing the number of parameters and the amount of calculation, while maintaining the important information of the features. The data is pooled, and the calculation expression of the pooling operation is as follows:

[0109] y(t)1=max(x(t),x(t+1),…,x(t+p-1))

[0110] Among them, p represents the size of the pooling window and y(t)1 represents the pooling output.

[0111] S22, based on step S21, establishing a long short-term memory network;

[0112] After the running data of the production line components are processed by the one-dimensional convolutional neural network 1D-CNN, the extracted feature data will enter the long short-term memory network LSTM, and the data will be gradually input into the LSTM network in the time series. For each time step, the input data x t The hidden state h of the previous time step t-1 The forget gate determines the forget information based on the current input and the previous hidden state, and calculates the forget gate output f through a Sigmoid activation function. t , whose value is between 0 and 1, represents the proportion of information that should be retained or forgotten. t The calculation expression is as follows:

[0113] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0114] Among them, W f and b f represents the weight and bias of the forget gate, σ represents the sigmoid activation function, and f t Used to adjust the previous unit state C t-1 , delete unnecessary information.

[0115] Set the input gate to decide which information in the current input should be stored in the cell state. It operates in two steps: first use the Sigmoid function to calculate which values ​​need to be updated, and then use the Tanh function to generate new candidate vectors Input gate output i t With candidate vector The calculation expression is as follows:

[0116] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0117]

[0118] Among them, W i represents the input gate weight matrix, b i represents the input gate bias, W c represents the weight matrix, b c A bias term representing the candidate state.

[0119] Input gate output i t Controls the new candidate vector How to add unit status. Updated unit status C t Determined by the calculation results of the forget gate output and the input gate, C t The long-term information required by the network is retained, and its calculation expression is as follows:

[0120]

[0121] Set the output gate to determine the output h of the LSTM unit t , through the current cell state C t With input o t Combined, calculated by Tanh and Sigmoid functions. The output gate controls the output information of the final LSTM unit, which is also the hidden state passed to the next time step. LSTM maintains and updates information in each time step of the time series, enabling it to capture long-term dependencies and thus make predictions or classifications more accurately. t With o t The calculation expression is as follows:

[0122] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0123] h t =o t *tanh(C t )

[0124] Among them, W o represents the output gate weight matrix, b o Represents the output gate bias.

[0125] S23, based on step S22, adding an attention mechanism to analyze the encoding features;

[0126] After the data is processed by 1D-CNN-LSTM, a fusion attention mechanism is added to assign different weights to different feature maps.

[0127] This embodiment effectively enhances the model's attention to global features by introducing a fusion attention mechanism (including a multi-head attention mechanism and an external attention mechanism). The multi-head attention mechanism can capture the multi-level features of input data from different dimensions, while the external attention mechanism improves the modeling ability of potential correlations between data sets by constructing a shared matrix, further enhancing the generalization and prediction accuracy of the model.

[0128] The fusion attention mechanism is based on the MultiHead Attention (MHA) mechanism and the External Attention (EA) mechanism. The attention mechanism is added to analyze the encoding features to improve the quality and representativeness of the encoding features in the network. The specific operation process is as follows:

[0129] 1) Establish a multi-head attention mechanism MHA;

[0130] The structure principle of the multi-head attention mechanism is as follows Figure 5 As shown in the figure, [W Q1 ,W Q2 ,…,W Qn ]、[W K1 ,W K2 ,…,W Kn ]、[W V1 ,W V2 ,…,W Vn ] represents multiple attention heads [head1,head2,…,head n ], Q i , K i、V i (i∈(1,n)) represents query, key and value respectively, [S1,S2,…,S n ] represents the attention score matrix, [P1,P2,…,P n ] represents the probability distribution converted from the attention score, [O1,O2,…,O n ] represents the attention probability distribution P i Sum value vector V i The output of the multiplied attention head, W o represents a linear transformation that combines the inputs of different attention heads.

[0131] MHA is an extension of the self-attention mechanism, which enables the model to pay attention to different parts of the input sequence. By calculating the similarity between signal channels to obtain its unique representation, different positions of the sequence can be associated to amplify the contribution of important features. Its basic form is as follows:

[0132]

[0133] Among them, Q (Query) represents the query matrix, K (Key) represents the key matrix, and d k Represents the dimensions of query and key, V(Value) represents the value matrix, Attention(Q,K,V) represents the output representation obtained by finding matching features from the key matrix K based on the query matrix Q and performing weighted summation on the value matrix V, and T represents the transpose of the matrix.

[0134] Different aspects and subspace features of the input are captured through multiple independent self-attention heads, and each attention head performs the above self-attention calculation. The calculation expression is as follows:

[0135] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O

[0136] Among them, MultiHead(Q,K,V) represents the output representation of multiple attention heads, W O Represents a linear transformation, merging the inputs for different heads; each head i Represents an independent attention function, and the calculation expression is as follows:

[0137] head i =Attention(QW i Q ,KW i K ,VW i V )

[0138] Among them, W i Q ,W i K and W i V The projection matrix representing the query, key, and value maps the input to different representation spaces. The subscript h of head represents a hyperparameter whose range is determined by the model and has no fixed limit. The output of each attention head is linearly transformed and concatenated together to obtain the output of the multi-head attention mechanism.

[0139] 2) Establish an external attention mechanism (EA);

[0140] The structural principle of the external attention mechanism is as follows Figure 6 As shown in Figure 2, due to the quadratic complexity of the self-attention mechanism, it is easy to ignore the potential correlation between different samples, while EA has linear complexity and can enhance the generalization ability of the model. The fusion attention mechanism sends the features processed by the SA module to the EA module to obtain the correlation expression of the entire data set.

[0141] First, construct the shared matrix M k and M v In order to focus on the potential correlation between different samples, they can be regarded as shared storage units of the entire dataset, and their parameters can be used for learning and are independent of the dataset. Then use M k For feature P∈R m×n The attention map between the dataset and the shared matrix is ​​processed, and then the softmax function is used to normalize the attention map to obtain the scoring matrix.

[0142] Where R represents a real number matrix, m represents the number of feature dimensions, and n represents the sequence length. The scoring matrix is ​​composed of M v Fine-tune and output the final features. The EA module uses M k and M v Implicitly process the data set. The process expression of the EA mechanism is as follows:

[0143]

[0144] A=M v (S)

[0145] Among them, P j represents the j-th feature output by the SA module, S represents the score matrix, N represents the input sequence length, and A represents the attention matrix.

[0146] The attention mechanism increases the weight of key features by re-weighting features, while suppressing the influence of irrelevant features. MHA captures important information in the feature sequence, and EA further extracts the relevance of the global data set and enhances the representativeness of the features. The processed features are used to train and infer fault diagnosis models to improve the diagnostic accuracy and robustness of the models.

[0147] S24, add dynamic cross entropy loss function DCE;

[0148] Existing deep learning models often perform poorly when dealing with sample imbalance problems, resulting in a decrease in the prediction performance of rare fault categories. To this end, this embodiment designs a dynamic cross entropy loss function (DCE), which adjusts the weight factor in real time so that the model can better focus on categories with fewer samples during training, thereby effectively improving the prediction accuracy of rare fault categories. The process expression of the DCE mechanism is as follows:

[0149]

[0150] Among them, y pre Represents the predicted label category, y true represents the actual label category, represents the imbalance factor at the tth training epoch, which is calculated based on the fault prediction result of the t-1th epoch and the initial imbalance factor. represents the initial imbalance factor, which represents the ratio of the number of samples of each fault category to the maximum category, n c represents the total number of samples, and c represents the total number of categories.

[0151] The parameters involved in the DCE mechanism include: training cycle T; batch size B; classifier parameter θ; the working process is as follows:

[0152] 1) Input data set A0 = [A1, A2, ..., A n ], the attention matrix A is split into small batches A of size B B , label y = [y1,y2,…,y n ].

[0153] in, represents the jth feature after feature transformation, y j Indicates a unique encoding format.

[0154] 2) Count the number of samples of each fault type in the training data set, and calculate the imbalance factor for each fault category based on the inverse ratio of the number of samples

[0155] in, It represents the relative rarity of the i-th type of fault. The rarer the type, the larger the factor.

[0156] 3) Through A B Compute the loss and gradients:

[0157]

[0158] in, Represents the derivative of the loss function with respect to the model.

[0159] 4) Use Adam optimizer to update the classifier parameters θ(θ,β1,β2,μ,m0,v0)

[0160]

[0161] Among them, β1, β2 represent the decay parameters used to control the momentum and gradient square estimation, μ represents the initial value of the learning rate, m0, v0 represent the initial values ​​of the first-order matrix and the second-order matrix estimation, m t , v t represents the estimated values ​​of momentum and squared gradient at the tth training iteration, represents the bias-corrected momentum and squared gradient estimates.

[0162] 5) Training is performed in multiple epochs, and model parameters are updated cyclically. At the beginning of the tth training epoch, the imbalance factor is updated using the prediction results of the previous epoch combined with the initial imbalance factor. For each fault category j, check the number of samples that are misclassified in the current epoch and obtain the labeling matrix M t [j],M t [j] indicates whether the j-th fault is misclassified in the current epoch.

[0163] The specific rules for updating the imbalance factor are as follows:

[0164] If M t [j] = 0, that is, no misclassification, then the imbalance factor remains unchanged, a t [j] = a t-1 [j];

[0165] If M t [j]≠0, that is, misclassification, then adjust the imbalance factor, a t [j]=M t [j]×a0[j], that is, enlarging or reducing the initial imbalance factor a0[j] based on the degree of misclassification, and dynamically adjusting the penalty intensity of the loss function on this category.

[0166] Among them, a t[j] represents the imbalance factor of the jth category in the tth round of training, which is dynamically updated as the training progresses, and a0[j] represents the initial imbalance factor of the jth category.

[0167] In summary, the method of the present invention solves the problem that the existing model is difficult to process high-dimensional sensor data and solves the imbalanced distribution of fault samples. A 1D-CNN-LSTM-Attention-DCE fault prediction model is proposed, which integrates a one-dimensional convolutional neural network (1D-CNN), a long short-term memory network (LSTM) and an attention mechanism. The model uses 1D-CNN to extract local features from time series data, and then captures long-term dependencies through LSTM, and finally strengthens the weight distribution of key features through the attention mechanism. The model is designed with a fusion attention module containing a multi-head self-attention mechanism (MHA) and an external attention mechanism (EA) to extract rich spatiotemporal features from sensor data, which comprehensively improves the fault prediction model's ability to process complex time series data, and can accurately predict faults within a certain time length range for the production line. At the same time, by applying the dynamic cross entropy loss function (DCE), the model is dynamically adjusted to focus on categories with a smaller number of samples, thereby improving the prediction accuracy.

[0168] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A deep fusion network production line fault prediction method based on deep learning, the specific steps are as follows: S1, collect SCADA system or sensor data of automatic production line, perform data preprocessing, feature extraction and fault prediction label construction; First, the relevant state quantities of the automatic production line operation are collected through sensors or SCADA systems, and the data is pre-processed, that is, sliding window processing; in, The relevant state quantities of the production line operation are collected through sensors or SCADA systems, namely SCADA system or sensor data, including: temperature, current, voltage, vibration, pressure, humidity, gas and liquid flow; the statistical information obtained by sliding window processing data includes: mean, median, sum, standard deviation, maximum and minimum values; Then, feature extraction is performed on the data processed by the sliding window, including: automatic production line fault logs, maintenance logs, component replacement records, and robot metadata; Finally, combined with the extracted features, fault prediction labels are constructed for the preprocessed data, and the operation data of multiple automatic production line equipment are collected, and the automatic production lines that perform the same work tasks are automatically produced; The process of constructing fault labels for preprocessed data is as follows: according to the fault time point, a time window is drawn for the collected data, and the production line-related component data falling into the time window corresponding to the fault point is marked as faulty, and the rest of the data is marked as normal; S2. Based on step S1, a deep fusion network 1D-CNN-LSTM-Attention-DCE model combining a one-dimensional convolutional network, a long short-term memory neural network, a fusion attention mechanism, and a dynamic cross entropy loss function DCE is established, i.e., a fault prediction model; First, a one-dimensional convolutional neural network is established, which can slide the convolution kernel on the one-dimensional space of the input data to extract features; secondly, a long short-term memory network is established to capture the long-term dependencies between data; thirdly, an attention mechanism is added to analyze the encoded features to improve the quality and representativeness of the encoded features in the network; finally, an imbalance factor is introduced in the dynamic cross entropy function to update the weights in real time according to the classification results of each epoch during model training; S3, inputting the data pre-processed in step S1 into the fault prediction model constructed in step S2 for training; To train the 1D-CNN-LSTM-Attention-DCE model, first input the processed sample data into the 1D-CNN module to extract local features, then input the intermediate variables processed by the swish activation function into the LSTM module to extract temporal features, and finally the data flows into the fusion attention module; S4. Perform fault prediction using the fault prediction model trained in step S3 to obtain a fault prediction result.

2. According to claim 1, a deep fusion network production line fault prediction method based on deep learning is characterized in that: The step S2 is specifically as follows: S21. Establish a one-dimensional convolutional neural network; Set (x1, x2, …, x n ) represents the time series input feature value, (o1,o2,…,o n ) represents the high-level eigenvalue obtained after the convolution operation; Where n represents the sequence length; the convolution kernel and input data of the one-dimensional convolutional neural network 1D-CNN are both one-dimensional, including: one-dimensional convolution layer and pooling layer; The convolution layer slides a fixed-size window over the input sequence; each convolution operation outputs the dot product of the elements in the window and the convolution kernel elements, and its calculation expression is as follows: Among them, x represents the input sequence, w represents the convolution kernel, t represents the time step, a represents the index variable, that is, the position of the sliding convolution window in the input signal x and the convolution kernel w, k represents the convolution kernel size, and y(t) represents the convolution output; After the data is convolved, it enters the pooling layer and performs a pooling operation on the data. The calculation expression of the pooling operation is as follows: y(t)1=max(x(t),x(t+1),…,x(t+p-1)) Where p represents the size of the pooling window, and y(t)1 represents the pooling output; S22, based on step S21, establishing a long short-term memory network; After the running data of the production line components are processed by the one-dimensional convolutional neural network 1D-CNN, the extracted feature data will enter the long short-term memory network LSTM, and the data will be gradually input into the LSTM network in the time series; for each time step, the input data x t The hidden state h of the previous time step t-1 are passed to the LSTM unit together; the forget gate determines the forget information based on the current input and the previous hidden state, and calculates the forget gate output f through a Sigmoid activation function t , whose value is between 0 and 1, indicating the proportion of information that should be retained or forgotten; f t The calculation expression is as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ) Among them, W f and b f represents the weight and bias of the forget gate, σ represents the sigmoid activation function, and f t Used to adjust the previous unit state C t-1 , delete unnecessary information; Set the input gate, first use the Sigmoid function to calculate the value to be updated, and then use the Tanh function to generate a new candidate vector Input gate output i t With candidate vector The calculation expression is as follows: i t =σ(W i ·[h t-1 ,x t ]+b i ) Among them, W i represents the input gate weight matrix, b i represents the input gate bias, W c represents the weight matrix, b c A bias term representing the candidate state; Input gate output i t Controls the new candidate vector How to add unit status; updated unit status C t Determined by the calculation results of the forget gate output and the input gate, C t The long-term information required by the network is retained, and its calculation expression is as follows: Set the output gate to determine the output h of the LSTM unit t , through the current cell state C t With input o t Combined, calculated by Tanh and Sigmoid functions; the output gate controls the output information of the final LSTM unit; h t With o t The calculation expression is as follows: the t =σ(W o ·[h t-1 ,x t ]+b o ) h t =o t *tanh(C t ) Among them, W o represents the output gate weight matrix, b o represents the output gate bias; S23, based on step S22, adding an attention mechanism to analyze the encoding features; After the data is processed by 1D-CNN-LSTM, a fusion attention mechanism is added to assign different weights to different feature maps; The fusion attention mechanism is based on the multi-head attention MHA mechanism and the external attention mechanism EA. The attention mechanism is added to analyze the encoding features. The specific operation process is as follows: 1) Establish a multi-head attention mechanism MHA; The basic form of MHA is as follows: Where Q represents the query matrix, K represents the key matrix, and d k represents the dimension of query and key, V represents the value matrix, Attention(Q,K,V) represents the output representation obtained by finding matching features from the key matrix K based on the query matrix Q and performing weighted summation on the value matrix V, and T represents the transpose of the matrix; Different aspects and subspace features of the input are captured by multiple independent self-attention heads, and each attention head performs self-attention calculation; its calculation expression is as follows: MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O Among them, MultiHead(Q,K,V) represents the output representation of multiple attention heads, W O Represents a linear transformation, merging the inputs for different heads; each head i Represents an independent attention function, and the calculation expression is as follows: head i =Attention(QW i Q ,KW i K ,VW i V ) Among them, W i Q ,W i K and W i V The projection matrix representing the query, key, and value maps the input to different representation spaces. The subscript h of head represents a hyperparameter whose range is determined by the model and has no fixed limit. The output of each attention head is linearly transformed and concatenated together to obtain the output of the multi-head attention mechanism. 2) Establish an external attention mechanism EA; The fusion attention mechanism sends the features processed by the SA module to the EA module. First, the shared matrix M is constructed k and M v To focus on the potential correlation between different samples; then use M k For feature P∈R m×n Processing is performed to obtain the attention mapping between the data set and the shared matrix, and then the attention mapping is normalized using the softmax function to obtain the scoring matrix; Among them, R represents a real number matrix, m represents the number of feature dimensions, and n represents the sequence length; the scoring matrix is ​​composed of M v Fine-tune and output the final features; the EA module passes M k and M v Implicitly processing data sets; the process expression of the EA mechanism is as follows: A=M v (S) Among them, P j represents the jth feature output by the SA module, S represents the score matrix, N represents the input sequence length, and A represents the attention matrix; S24, add dynamic cross entropy loss function DCE; The process expression of the DCE mechanism is as follows: Among them, y pre Represents the predicted label category, y true represents the actual label category, represents the imbalance factor at the tth training epoch, which is calculated based on the fault prediction result of the t-1th epoch and the initial imbalance factor. represents the initial imbalance factor, which represents the ratio of the number of samples of each fault category to the maximum category, n c represents the total number of samples, and c represents the total number of categories; The parameters involved in the DCE mechanism include: training cycle T; batch size B; classifier parameter θ; the working process is as follows: 1) Input data set A0 = [A1, A2, ..., A n ], the attention matrix A is split into small batches A of size B B , label y = [y1,y2,…,y n ]; in, (j=1,2...n), represents the jth feature after feature transformation, y j Indicates a unique encoding format; 2) Count the number of samples of each fault type in the training data set, and calculate the imbalance factor for each fault category based on the inverse ratio of the number of samples in, (i=1,2...c) represents the relative rarity of the i-th type of fault. The rarer the type, the larger the factor. 3) Through A B Compute the loss and gradients: in, Represents the derivative of the loss function with respect to the model; 4) Use Adam optimizer to update the classifier parameters θ(θ,β1,β2,μ,m0,v0) Among them, β1, β2 represent the decay parameters used to control the momentum and gradient square estimation, μ represents the initial value of the learning rate, m0, v0 represent the initial values ​​of the first-order matrix and the second-order matrix estimation, m t , v t represents the estimated values ​​of momentum and squared gradient at the tth training iteration, represents the bias-corrected momentum and squared gradient estimates; 5) Training is performed in multiple epochs, and model parameters are updated cyclically. At the beginning of the tth training epoch, the imbalance factor is updated using the prediction results of the previous epoch combined with the initial imbalance factor. For each fault category j, check the number of samples that are misclassified in the current epoch and obtain the labeling matrix M t [j],M t [j] indicates whether the j-th fault is misclassified in the current epoch; The specific rules for updating the imbalance factor are as follows: If M t [j] = 0, that is, no misclassification, then the imbalance factor remains unchanged, a t [j] = a t-1 [j]; If M t [j]≠0, that is, misclassification, then adjust the imbalance factor, a t [j]=M t [j]×a0[j], that is, enlarging or reducing the initial imbalance factor a0[j] based on the degree of misclassification, and dynamically adjusting the penalty intensity of the loss function on this category; Among them, a t [j] represents the imbalance factor of the jth category in the tth round of training, which is dynamically updated as the training progresses, and a0[j] represents the initial imbalance factor of the jth category.

Images