Intelligent fault diagnosis method, device and equipment based on deep learning

Abstract

The invention relates to an intelligent fault diagnosis method, device and equipment based on deep learning. The method comprises the following steps: acquiring signal data, multi-channel data and multi-modal data of target equipment or parts; performing periodic sliding sampling on the signal data and the multi-channel data, and constructing an intelligent fault diagnosis model; a data set generated through sampling is input into a multi-channel data fusion layer through an input layer to be spliced and fused in the channel dimension, feature extraction is conducted on shallow layer features according to a hidden layer, and obtained deep layer features and multi-modal data are input into a multi-modal data fusion layer to be subjected to data fusion; fault classification is carried out on the fused features in a connection layer according to a classification function, and a loss function is set according to a classification prediction result and a label of a corresponding sample in a data set; and training the intelligent fault diagnosis model by using the loss function, and realizing intelligent fault diagnosis according to the trained intelligent fault diagnosis model. By adopting the method, the fault diagnosis accuracy of the intelligent equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an intelligent fault diagnosis method, device and equipment based on deep learning. Background Art

[0002] In the production process of modern industry, the damage and failure of production equipment or components are often accompanied. In addition, the working conditions are complex and harsh. The frequency of such failures is much higher than that of traditional production processes. Therefore, in order to reduce production risks, timely discover and eliminate faults, and avoid further losses, fault diagnosis technology has been proposed and widely used in the production process. Traditional fault diagnosis is mainly based on signal processing technology. The fault diagnosis technology at this stage analyzes the relevant signal data of the target equipment or components during operation and finds fault features from them to achieve fault diagnosis, such as wavelet analysis, Bayesian decision making, probability statistics-based time series model diagnosis and fault tree analysis, etc. However, traditional fault diagnosis technology often requires a lot of prior knowledge and is difficult to process large-scale complex data. Therefore, some data-driven methods have emerged, such as diagnostic methods based on statistical machine learning. These methods, such as support vector machines, random forests, and decision trees, can automatically extract fault features from signal data to a certain extent, and have also been verified on data of a certain scale and complexity. However, these methods are difficult to automatically process high-dimensional features with a certain depth and complexity in the data, and have poor scalability and adaptability. With the continuous enhancement of hardware performance in recent years, deep learning, a data-driven method with powerful fitting capabilities, has gradually come into people's view.

[0003] Deep Learning uses multi-layer neural networks for learning and representation, aiming to enable machines to have analytical learning capabilities like humans by simulating the working mode of human brain neural networks. It adjusts the weight parameters of the neural network model through training with a large amount of data to explore the deep laws and features in the data, fit its internal laws, and further generalize to other data. Since deep learning does not require much prior knowledge, has good scalability and adaptability, and its powerful fitting ability enables it to automatically extract features from data and analyze them, it has gained favor in the field of intelligent fault diagnosis.

[0004] However, deep learning often assumes that the same distribution exists in the training data and the data to be generalized, that is, the training data and the data to be generalized have the same internal rules. Although the rules cannot be expressed by formulas, they can be approximated with infinite precision through the fitting ability of deep learning, so that the trained model can also perform well on the data to be generalized. However, in the modern industrial production process, various working conditions are intertwined and very complex. There may be different distribution rules in the data under different working conditions, and it is often very costly to obtain fault data under certain working conditions. Therefore, transfer learning is introduced for cross-condition diagnosis.

[0005] Transfer learning can transfer the knowledge learned in the source domain data to the target domain data by comparing the source domain data and the target domain data. Through transfer learning, the neural network model can be trained on only complete source domain data and unlabeled target domain data, and the model can perform well in both the source domain and the target domain, which makes transfer learning widely used in cross-condition diagnosis. However, transfer learning requires data from both the source domain and the target domain when training the model. If there is no data in the target domain, then transfer learning may fail. In modern industrial production processes, the cost of collecting data under certain harsh production conditions is very high, or it is technically difficult to be feasible.

[0006] In addition, most previous deep learning-based fault diagnosis technologies ignored the periodicity of the target equipment's operation, resulting in low fault diagnosis accuracy. Summary of the invention

[0007] Based on this, it is necessary to provide a deep learning-based intelligent fault diagnosis method, device and equipment that can improve the accuracy of fault diagnosis of intelligent devices in response to the above technical problems.

[0008] An intelligent fault diagnosis method based on deep learning, the method comprising:

[0009] Acquire signal data, multi-channel data and multi-modal data of the target device or component; signal data is the time series signal sampled by the sensor directly from the target device at a certain frequency within a certain period of time; multi-channel data is the data collected by other sensors for other components related to the target device or component; multi-modal data is the data of other modes related to the target device or component captured by the sensor;

[0010] Perform periodic sliding sampling on signal data and multi-channel data to obtain a data set generated by sampling;

[0011] Construct an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer and a fully connected layer;

[0012] The data set generated by sampling is input into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; the shallow features are extracted from the deep features according to the hidden layer, and the obtained deep features and multimodal data are input into the multimodal data fusion layer for data fusion to obtain fused features; the fused features are classified into faults according to the classification function in the connection layer to obtain the classification prediction results;

[0013] A loss function is set according to the classification prediction results and the labels of the corresponding samples in the data set; the intelligent fault diagnosis model is trained using the loss function to obtain a trained intelligent fault diagnosis model; and intelligent fault diagnosis is implemented according to the trained intelligent fault diagnosis model.

[0014] An intelligent fault diagnosis device based on deep learning, the device comprising:

[0015] The data acquisition module is used to acquire the signal data, multi-channel data and multi-modal data of the target device or component; the signal data is the time series signal sampled by the sensor directly from the target device at a certain frequency within a certain time; the multi-channel data is the data collected by other sensors for other components related to the target device or component; the multi-modal data is the data of other modes related to the target device or component captured by the sensor;

[0016] The data sampling and model building module is used to perform periodic sliding sampling on signal data and multi-channel data to obtain a data set generated by sampling; build an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer and a fully connected layer;

[0017] The fault classification prediction module is used to input the sampled data set into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; perform deep feature extraction on the shallow features according to the hidden layer, input the obtained deep features and multi-modal data into the multi-modal data fusion layer for data fusion to obtain fused features; perform fault classification on the fused features according to the classification function in the connection layer to obtain classification prediction results;

[0018] The model training and fault diagnosis module is used to set the loss function according to the classification prediction results and the labels of the corresponding samples in the data set; use the loss function to train the intelligent fault diagnosis model to obtain a trained intelligent fault diagnosis model; and implement intelligent fault diagnosis based on the trained intelligent fault diagnosis model.

[0019] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0020] Acquire signal data, multi-channel data and multi-modal data of the target device or component; signal data is the time series signal sampled by the sensor directly from the target device at a certain frequency within a certain period of time; multi-channel data is the data collected by other sensors for other components related to the target device or component; multi-modal data is the data of other modes related to the target device or component captured by the sensor;

[0021] Perform periodic sliding sampling on signal data and multi-channel data to obtain a data set generated by sampling;

[0022] Construct an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer and a fully connected layer;

[0023] The data set generated by sampling is input into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; the shallow features are extracted from the deep features according to the hidden layer, and the obtained deep features and multimodal data are input into the multimodal data fusion layer for data fusion to obtain fused features; the fused features are classified into faults according to the classification function in the connection layer to obtain the classification prediction results;

[0024] A loss function is set according to the classification prediction results and the labels of the corresponding samples in the data set; the intelligent fault diagnosis model is trained using the loss function to obtain a trained intelligent fault diagnosis model; and intelligent fault diagnosis is implemented according to the trained intelligent fault diagnosis model.

[0025] The above-mentioned intelligent fault diagnosis method, device and equipment based on deep learning, this application adopts the signal data, multi-channel data and multi-modal data of the target device or component as input, which contains the periodicity of the fault characteristics, and uses the periodicity to determine the sample length in the data set. While ensuring the best possible diagnostic performance, it can also reduce the number of model parameters to achieve a lighter model. At the same time, the sensors configured on the smart device not only collect the relevant signal data of the target device or component operation, but also collect other data indirectly related to the target device or component. These data usually contain auxiliary information that is conducive to improving the diagnostic accuracy, and this information can also be used to improve the performance of fault diagnosis. Then, the signal data and multi-channel data are periodically sampled, and an intelligent fault diagnosis model is constructed. The generated data set is sent to the multi-channel data fusion layer to extract the fused shallow features, and the signal data and the multi-channel data are spliced ​​in the channel dimension. Then, a convolutional neural network is used and two-dimensional convolution is introduced to extract the shallow fusion features. By applying two-dimensional convolution, the convolution kernel can span multiple channels and enhance the field of view of the convolution kernel, so as to integrate the data of multiple channels for feature extraction, and then the extracted shallow features are sent to the hidden layer to realize the extraction of deep features. The attention mechanism is applied to the multimodal data to highlight the parts that are beneficial to the diagnosis results and suppress the parts that are useless for diagnosis. These highlighted useful features are spliced ​​with the deep features extracted by the hidden layer before to realize multimodal data fusion. The data is sent to the fully connected layer for classification, and the probability values ​​of each fault type to which the sample belongs are output. The training loss is calculated and back propagation is performed to adjust the weights to realize model training. Intelligent fault diagnosis based on the trained intelligent fault diagnosis model can greatly improve the accuracy of intelligent fault diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of an intelligent fault diagnosis method based on deep learning in one embodiment;

[0027] Figure 2 A schematic diagram of a framework of an intelligent fault diagnosis model in one embodiment;

[0028] Figure 3 A schematic diagram of a periodic sliding sampling method in an embodiment;

[0029] Figure 4 A schematic diagram of a multi-channel data fusion method in another embodiment;

[0030] Figure 5 A schematic diagram of a multimodal data fusion method in one embodiment;

[0031] Figure 6 A schematic diagram of the change in model accuracy with and without a multi-channel data fusion method in one embodiment;

[0032] Figure 7 A schematic diagram of the change in model accuracy with and without a multimodal data fusion method in one embodiment;

[0033] Figure 8 The performance indicators of the present application on two public data sets in one embodiment are shown in FIG. 1 ; (a) is a test set accuracy effect diagram; (b) is a confusion matrix effect diagram;

[0034] Fig. 9 is a structural block diagram of an intelligent fault diagnosis device based on deep learning in one embodiment;

[0035] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] In one embodiment, Figure 1 As shown, an intelligent fault diagnosis method based on deep learning is provided, comprising the following steps:

[0038] Step 102, acquiring signal data, multi-channel data and multi-modal data of the target device or component; the signal data is a timing signal sampled by a sensor directly from the target device at a certain frequency within a certain time; the multi-channel data is data collected by other sensors for other components related to the target device or component; the multi-modal data is data of other modes related to the target device or component captured by the sensor.

[0039] This application uses three types of data for input, namely: signal data of the target device or component, multi-channel data, and multi-modal data. Among them, the signal data is a time series signal sampled directly from the target device by the sensor at a certain frequency within a certain period of time. It is data sampling directly for the target device, which can be described as follows:

[0040]

[0041] Among them, X is the sampled signal data, X 0 ,X 1 ,…,X N-1It is a data point in the signal data. Multi-channel data is data collected by other sensors for other components related to the target device or component, such as the housing or motor current connected to the target device or component. These data have the same structure and data format as the signal data, which can be described as follows:

[0042]

[0043] Among them, X mc is multi-channel data, C is the number of channels, It is a data point of multi-channel data. Multimodal data is data of other modes related to the target device or component captured by the sensor, such as load, temperature, etc. These data may have different structures and data formats from the above two data, which can be described as follows:

[0044]

[0045] Among them, X mm For multimodal data, Other modal data sampled by different sensors may have different data forms and structures.

[0046] The periodicity of fault characteristics reflected in the signal data can be used to determine the sample length in the data set, which can reduce the number of model parameters while ensuring the best possible diagnostic performance to achieve a lighter model. Furthermore, in modern industrial production processes, a large number of sensors are often deployed in workshops. These sensors not only collect signal data related to the operation of the target equipment or components, but also collect other data indirectly related to the target equipment or components. These data usually contain auxiliary information that is conducive to improving diagnostic accuracy. This information can be used to improve the performance of fault diagnosis.

[0047] Step 104, perform periodic sliding sampling on the signal data and the multi-channel data to obtain a data set generated by sampling; construct an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multimodal data fusion layer and a fully connected layer.

[0048] like Figure 2 The figure shows the overall process framework of the present application, including the input layer, multi-channel data fusion layer, hidden layer, multi-modal data fusion layer and output layer. In order to make full use of the data collected by multiple sensors, the present application uses the signal data, multi-channel data and multi-modal data of the target device or component to perform intelligent fault diagnosis for the target device or component. First, periodic sliding sampling is performed on the signal data and multi-channel data, such as Figure 3 First, calculate the number of sampling points of the target device or component in one operation cycle:

[0049]

[0050] Where n is the number of sampling points of the target device or component in one operating cycle, f is the sampling frequency of the sensor, and RPM is the number of cycles per minute of the target device or component. Then, the sample length of the data set to be generated is determined according to the number of sampling points of the target device or component in one cycle:

[0051] L = α × n, α∈(0,1]

[0052] Where L is the sample length of the data set to be generated, n is the number of sampling points of the target device or component in one operation cycle, and α is the scaling factor. And the sliding step size is:

[0053]

[0054] Where stride is the sliding step size, and L is the sample length of the data set to be generated. The data set generated after sampling is:

[0055]

[0056] Among them, X pss is the generated data set, X[a,b) represents a continuous subinterval data [X a ,X a+1 ,…,X b-1 ], the whole sampling process is as follows Figure 3 The data set generated by sampling through this method can not only make full use of the information of signal data and multi-channel data to generate the data set, but also ensure that each sample has sufficient fault information, and the sample length is as short as possible, thereby reducing model parameters and speeding up training and reasoning.

[0057] Step 106, input the sampled data set into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; extract deep features from the shallow features according to the hidden layer, input the obtained deep features and multimodal data into the multimodal data fusion layer for data fusion to obtain fused features; classify the fused features according to the classification function in the connection layer to obtain classification prediction results.

[0058] The signal data and multi-channel data after periodic sliding sampling are sent to the multi-channel data fusion layer for data fusion. First, the signal data is taken as the 0th channel and the multi-channel data is spliced ​​in the channel dimension to obtain:

[0059]

[0060] Among them, Concat C (X,X mc ) means concatenating the signal data with the multi-channel data in the channel dimension. is the signal data, and the rest is multi-channel data. Then it is sent to the Convolutional Neural Network (CNN), and two-dimensional convolution is applied. The number of channels C+1 is regarded as the height, the sample length L is regarded as the width, and the convolution kernel with a height and width of (C+1)×1 is used for convolution to extract shallow fusion features, and the output is:

[0061]

[0062] Among them, k 0 ,k 1 ,…,k C is the convolution kernel component, that is, the trainable parameter in the convolutional neural network. The process is as follows Figure 4 By applying two-dimensional convolution, the convolution kernel can span multiple channels, enhancing the field of view of the convolution kernel, thereby integrating data from multiple channels for feature extraction.

[0063] The extracted shallow features are sent to the hidden layer to extract deep features. The hidden layer is implemented with a fully convolutional neural network, and a residual connection is added. The residual connection only performs channel transformation on the input, and its convolution kernel length and step size are both 1. Through the residual connection, the model performance will not degrade when the network is deepened, which enhances the scalability and adaptability of the network. A residual connection and the convolutional neural network layer it includes constitute a residual convolution block, and the hidden layer is composed of several residual convolution blocks. In the convolutional neural network, a normalization layer and activation function are used after each convolution layer. The normalization layer is implemented using Instance Normalization: the mean μ and variance σ are calculated along the channel, and then the data is normalized. And by adding scaling and translation y=γx′+β, the output of each layer can be normalized to return to a fixed mean and variance, which makes each layer of the neural network not strongly dependent on the output of the previous layer, enhances the robustness of the model and accelerates convergence. The activation function is implemented using the GELU function:

[0064]

[0065] in, There is also a downsampling layer after the activation function of the first convolutional layer and the last convolutional layer, which is implemented using a convolutional neural network. Using a convolutional neural network to implement downsampling helps to retain information without directly discarding information. The convolution kernel length and the moving step size are both 2.

[0066] After that, the deep features extracted from the hidden layer and other modal data are sent to the multimodal data fusion layer to fuse the deep features with other modal data. Send it to a multi-layer perceptron to find the similarity weight:

[0067]

[0068] Among them, fea is the deep feature extracted by the hidden layer. For other modal data, is the similarity weight between the extracted deep features and other modal data. Then it is normalized by softmax:

[0069]

[0070] Among them, a i is the normalized weight corresponding to other modal data, and e is a natural constant. Then, by multiplying the weight with the corresponding multimodal data, the attention value can be obtained:

[0071]

[0072] Among them, Attention i is the attention value corresponding to other modal data. By using the attention mechanism, the content in other modal data that is helpful for diagnostic performance can be highlighted, while the content that is not helpful for the diagnostic results can be suppressed, so that the model can focus more on the truly effective part of the data. Finally, the above attention value is concatenated with the deep features to obtain the features after fusion with multimodal data, which is the input of the fully connected classification network:

[0073] Concat(fea,Attrntion 1 ,…,Attention N′ )

[0074] Among them, fea is the deep feature extracted by the hidden layer. The whole process is as follows Figure 5 shown.

[0075] Finally, the above fused feature data is sent to the fully connected layer for classification. Considering the classification function in is the prediction result, F(·) is the function fitted by the network, x is the input of the classification network, and the prediction result is a probability distribution. are the probabilities that the sample data belongs to fault type 1, 2, ..., n respectively, and the fault type with the largest probability value is taken as the classification result, that is:

[0076]

[0077] Among them, i=1,2,…,n.

[0078] Step 108, setting a loss function according to the classification prediction results and the labels of the corresponding samples in the data set; using the loss function to train the intelligent fault diagnosis model to obtain a trained intelligent fault diagnosis model; and implementing intelligent fault diagnosis according to the trained intelligent fault diagnosis model.

[0079] The data is sent to the fully connected classification network for classification, and the probability values ​​of each fault type to which the sample belongs are output. The training loss is calculated and back propagation is performed to adjust the weights until all the training data are trained, which is an epoch. After several epochs, when the loss no longer decreases, the model reaches a convergence state, and the training is completed at this time. Intelligent fault diagnosis based on the trained intelligent fault diagnosis model can greatly improve the accuracy of intelligent fault diagnosis.

[0080] In the above-mentioned intelligent fault diagnosis method based on deep learning, the present application adopts the signal data, multi-channel data, and multi-modal data of the target device or component as input, which contains the periodicity of the fault characteristics, and uses the periodicity to determine the sample length in the data set. While ensuring the best possible diagnostic performance, it can also reduce the number of model parameters to achieve a lighter model. At the same time, the sensors configured on the smart device not only collect relevant signal data of the target device or component operation, but also collect other data indirectly related to the target device or component. These data usually contain auxiliary information that is conducive to improving the diagnostic accuracy, and this information can also be used to improve the performance of fault diagnosis. Then, the signal data and multi-channel data are periodically sampled, and an intelligent fault diagnosis model is constructed. The generated data set is sent to the multi-channel data fusion layer to extract the fused shallow features, and the signal data and the multi-channel data are spliced ​​in the channel dimension. Then, a convolutional neural network is used and two-dimensional convolution is introduced to extract the shallow fusion features. By applying two-dimensional convolution, the convolution kernel can span multiple channels and enhance the field of view of the convolution kernel, so as to integrate the data of multiple channels for feature extraction, and then the extracted shallow features are sent to the hidden layer to realize the extraction of deep features. The attention mechanism is applied to the multimodal data to highlight the parts that are beneficial to the diagnosis results and suppress the parts that are useless for diagnosis. These highlighted useful features are spliced ​​with the deep features extracted by the hidden layer before to realize multimodal data fusion. The data is sent to the fully connected layer for classification, and the probability values ​​of each fault type to which the sample belongs are output. The training loss is calculated and back propagation is performed to adjust the weights to realize model training. Intelligent fault diagnosis based on the trained intelligent fault diagnosis model can greatly improve the accuracy of intelligent fault diagnosis.

[0081] In one embodiment, periodic sliding sampling is performed on signal data and multi-channel data to obtain a data set generated by sampling, including:

[0082] Calculate the number of sampling points of the target device or component in one operation cycle as:

[0083]

[0084] Where n is the number of sampling points of the target device or component in one operating cycle, f is the sampling frequency of the sensor, and RPM is the number of cycles per minute of the target device or component;

[0085] The sample length of the data set to be generated is calculated based on the number of sampling points:

[0086] L = α × n, α∈(0,1]

[0087] Where L is the sample length of the data set to be generated, n is the number of sampling points, and α is the scaling factor;

[0088] The sliding step length is determined by using the sample length:

[0089]

[0090] The signal data and multi-channel data are periodically sampled according to the sliding step size, and the data set generated by the sampling is:

[0091]

[0092] Among them, X pss is the generated data set, X[a,b) represents a continuous subinterval data [X a ,X a+1 ,…,X b-1 ].

[0093] In one embodiment, the data set generated by sampling is input into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features, including:

[0094] The data set generated by sampling is input into the multi-channel data fusion layer through the input layer, and is spliced ​​with the 0th channel and the multi-channel data in the channel dimension. The spliced ​​data is:

[0095]

[0096] Among them, Concat C (X,X mc ) means concatenating the signal data with the multi-channel data in the channel dimension. is signal data, and the other parts are multi-channel data;

[0097] The convolutional neural network is used to regard the number of channels C+1 as the height and the sample length L as the width, and a convolution kernel with a height and width of (C+1)×1 is applied to extract shallow fusion features, and the shallow features are obtained as follows:

[0098]

[0099] Among them, k 0 ,k 1 ,…,k C is the convolution kernel component, that is, the trainable parameter in the convolutional neural network.

[0100] In one embodiment, the hidden layer includes a plurality of residual convolution blocks; the residual convolution block is composed of a residual connection layer and a convolutional neural network layer included therein; and deep feature extraction is performed on shallow features according to the hidden layer, including:

[0101] According to the residual connection layer, the shallow features are transformed into channels, and then the features are extracted according to the convolutional layer in the convolutional neural network, and then normalized, activated and downsampled to obtain the deep features; the activation function used in the activation process is:

[0102]

[0103] in, x represents the data output by the previous layer of neural network.

[0104] In one embodiment, the obtained deep features and multimodal data are input into a multimodal data fusion layer for data fusion to obtain fused features, including:

[0105] The deep features and multimodal data are fed into a multilayer perceptron to obtain the similarity weights:

[0106]

[0107] Among them, fea is the deep feature extracted by the hidden layer. It is multimodal data.

[0108] In one embodiment, the similarity weight is normalized to obtain the normalized weight:

[0109]

[0110] Multiply the normalized weights by the corresponding multimodal data to obtain the attention value of the corresponding multimodal data:

[0111]

[0112] In one embodiment, the attention value and deep features of the corresponding multimodal data are concatenated to obtain the fused features:

[0113] Concat(fea,Attrntion 1 ,…,Attention N′ ).

[0114] In one embodiment, the loss function is set according to the classification prediction result and the label of the corresponding sample in the data set:

[0115]

[0116] Among them, y i is the probability value at the probability label position i expressed in one-hot form, is the probability value of the predicted result at position i.

[0117] In a specific embodiment, in order to evaluate the performance of the present application, the present application uses the CWRU dataset and the JNU dataset for simulation and performance testing. The Adam optimizer is used, the learning rate is set to 0.01, and 100 epochs are iterated on the two datasets respectively. Figure 6 The comparison of the accuracy changes of the models on the test set during the training process is given in two cases: with and without the multi-channel data fusion method. The model without the multi-channel data fusion method is processed using the classic one-dimensional convolution. It can be seen that during the entire training process, the accuracy of the model with the multi-channel data fusion method is often higher than that of the model without the multi-channel data fusion method, and it is maintained throughout the training period, which shows the effectiveness of the proposed multi-channel data fusion method. In addition, Figure 7 A comparison of the accuracy of the model with the multimodal data fusion method and the model without the multimodal data fusion method on the test set during the training process is given. It can be seen that between the two corresponding methods on the corresponding data sets, the accuracy of the model with the multimodal data fusion method on the test set during the training process is higher than that of the model without the multimodal data fusion method, and this advantage is gradually determined until the final model loss converges and the model training is completed. Therefore, the effectiveness of the proposed multimodal data fusion method can be proved. In addition, it is also necessary to note that the working conditions of the data set used in training are not consistent with the working conditions of the data set used in testing, and the training results reflect that the model trained by this application can be well generalized to the data of the target working conditions, which shows that the invariant domain attributes existing in multiple domains are found by the multimodal data fusion method, which enables this application to overcome the defects of transfer learning and generalize the model from the source domain data to the target domain data in the absence of target domain data.

[0118] at last, Figure 8 The performance of this application on two public datasets is shown, including the accuracy changes on the test set during training and the confusion matrix, such as Figure 8 (a) and Figure 8 As shown in (b), through the display and explanation of these performance indicators, it can be proved that the proposed method performs well in actual situations. Compared with some other intelligent fault diagnosis methods, the solution proposed in this application is better.

[0119] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0120] In one embodiment, Fig. 9 As shown, an intelligent fault diagnosis device based on deep learning is provided, including: a data acquisition module 902, a data sampling and model building module 904, a data sampling and model building module 906 and a model training and fault diagnosis module 908, wherein:

[0121] The data acquisition module 902 is used to acquire signal data, multi-channel data and multi-modal data of the target device or component; the signal data is a time series signal sampled by a sensor directly from the target device at a certain frequency within a certain time; the multi-channel data is data collected by other sensors for other components related to the target device or component; the multi-modal data is data of other modes related to the target device or component captured by the sensor;

[0122] The data sampling and model building module 904 is used to perform periodic sliding sampling on the signal data and the multi-channel data to obtain a data set generated by sampling; and to build an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer, and a fully connected layer;

[0123] The fault classification prediction module 906 is used to input the data set generated by the sampling into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; perform deep feature extraction on the shallow features according to the hidden layer, input the obtained deep features and multi-modal data into the multi-modal data fusion layer for data fusion to obtain fused features; perform fault classification on the fused features according to the classification function in the connection layer to obtain classification prediction results;

[0124] The model training and fault diagnosis module 908 is used to set a loss function according to the classification prediction results and the labels of the corresponding samples in the data set; use the loss function to train the intelligent fault diagnosis model to obtain a trained intelligent fault diagnosis model; and implement intelligent fault diagnosis according to the trained intelligent fault diagnosis model.

[0125] For the specific limitations of the intelligent fault diagnosis device based on deep learning, please refer to the limitations of the intelligent fault diagnosis method based on deep learning above, which will not be repeated here. Each module in the above-mentioned intelligent fault diagnosis device based on deep learning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0126] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an intelligent fault diagnosis method based on deep learning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0127] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An intelligent fault diagnosis method based on deep learning, characterized in that: The method comprises: Acquire signal data, multi-channel data and multi-modal data of the target device or component; the signal data is a time series signal sampled by a sensor directly from the target device at a certain frequency within a certain time; the multi-channel data is data collected by other sensors for other components related to the target device or component; the multi-modal data is data of other modes related to the target device or component captured by the sensor; Performing periodic sliding sampling on the signal data and the multi-channel data to obtain a data set generated by sampling; Constructing an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer and a fully connected layer; The data set generated by the sampling is input into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; deep features are extracted from the shallow features according to the hidden layer, and the obtained deep features and multimodal data are input into the multimodal data fusion layer for data fusion to obtain fused features; fault classification is performed on the fused features according to the classification function in the connection layer to obtain classification prediction results; A loss function is set according to the classification prediction results and the labels of the corresponding samples in the data set; the intelligent fault diagnosis model is trained using the loss function to obtain a trained intelligent fault diagnosis model; and intelligent fault diagnosis is implemented according to the trained intelligent fault diagnosis model.

2. The method according to claim 1, characterized in that The signal data and the multi-channel data are periodically sampled to obtain a data set generated by sampling, including: Calculate the number of sampling points of the target device or component in one operation cycle as: Where n is the number of sampling points of the target device or component in one operating cycle, f is the sampling frequency of the sensor, and RPM is the number of cycles per minute of the target device or component; The sample length of the data set to be generated is calculated according to the number of sampling points: L = α × n, α∈(0,1] Where L is the sample length of the data set to be generated, n is the number of sampling points, and α is the scaling factor; The sliding step length is determined using the sample length: According to the sliding step length, the signal data and the multi-channel data are periodically sampled, and the data set generated by the sampling is obtained as follows: Among them, X pss is the generated data set, X[a,b) represents a continuous subinterval data [X a ,X a+1 ,…,X b-1 ].

3. The method according to claim 1, characterized in that The data set generated by the sampling is input into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features, including: The data set generated by the sampling is input into the multi-channel data fusion layer through the input layer, and is spliced ​​with the 0th channel and the multi-channel data in the channel dimension to obtain the spliced ​​data: Among them, Concat C (X,X mc ) means concatenating the signal data with the multi-channel data in the channel dimension. is signal data, and the other parts are multi-channel data; The convolutional neural network is used to regard the number of channels C+1 as the height and the sample length L as the width, and a convolution kernel with a height and width of (C+1)×1 is applied to extract shallow fusion features, and the shallow features are obtained as follows: Among them, k0, k1, …, k C is the convolution kernel component, that is, the trainable parameter in the convolutional neural network.

4. The method according to claim 1, characterized in that: The hidden layer includes a plurality of residual convolution blocks; the residual convolution block is composed of a residual connection layer and a convolutional neural network layer included therein; and deep feature extraction is performed on the shallow features according to the hidden layer, including: The shallow features are transformed through channels according to the residual connection layer, and then the features are extracted according to the convolution layer in the convolutional neural network, and then normalized, activated and downsampled to obtain deep features; the activation function used in the activation process is: in, x represents the data output by the previous layer of neural network.

5. The method according to claim 1, characterized in that The obtained deep features and multimodal data are input into the multimodal data fusion layer for data fusion to obtain the fused features, including: The deep features and multimodal data are fed into a multilayer perceptron to obtain the similarity weight: Among them, fea is the deep feature extracted by the hidden layer. It is multimodal data.

6. The method according to claim 5, characterized in that The method further comprises: The similarity weight is normalized to obtain the normalized weight: The normalized weight is multiplied by the corresponding multimodal data to obtain the attention value of the corresponding multimodal data:

7. The method according to claim 6, characterized in that Said also includes: The attention value and deep features of the corresponding multimodal data are concatenated to obtain the fused features: Concat(fea,Attrntion1,…,Attention N′ )。 8. The method according to claim 1, characterized in that The loss function is set according to the classification prediction results and the labels of the corresponding samples in the data set: Among them, y i is the probability value at the probability label position i expressed in one-hot form, is the probability value of the predicted result at position i.

9. An intelligent fault diagnosis device based on deep learning, characterized in that: The device comprises: A data acquisition module, used to acquire signal data, multi-channel data and multi-modal data of a target device or component; the signal data is a time series signal sampled by a sensor directly from the target device at a certain frequency within a certain period of time; the multi-channel data is data collected by other sensors for other components related to the target device or component; the multi-modal data is data of other modes related to the target device or component captured by the sensor; A data sampling and model building module is used to perform periodic sliding sampling on the signal data and multi-channel data to obtain a data set generated by sampling; and to build an intelligent fault diagnosis model; the intelligent fault diagnosis model includes an input layer, a multi-channel data fusion layer, a hidden layer, a multi-modal data fusion layer, and a fully connected layer; A fault classification prediction module is used to input the data set generated by the sampling into the multi-channel data fusion layer through the input layer for splicing and fusion in the channel dimension to obtain shallow features; perform deep feature extraction on the shallow features according to the hidden layer, input the obtained deep features and multimodal data into the multimodal data fusion layer for data fusion to obtain fused features; perform fault classification on the fused features according to the classification function in the connection layer to obtain classification prediction results; The model training and fault diagnosis module is used to set a loss function according to the classification prediction results and the labels of the corresponding samples in the data set; use the loss function to train the intelligent fault diagnosis model to obtain a trained intelligent fault diagnosis model; and implement intelligent fault diagnosis according to the trained intelligent fault diagnosis model.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images