Coal mine transmission belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite

Abstract

The invention belongs to belt fault diagnosis of a coal mine transmission belt, and particularly relates to a fault diagnosis method based on MobileNetV2 and TensorFlow Lite. Comprising the steps of preprocessing data, constructing a MobileNetV2 model, improving a global pooling layer, adding a full connection layer, improving an output layer, improving a loss function and the like. Image data are collected and preprocessed, a model structure is constructed and improved, a dynamic attention mechanism and multi-task learning are introduced, and a loss function is improved in combination with the multi-task learning and the attention mechanism, so that the robustness and generalization ability of the model are improved, high-precision recognition and real-time diagnosis of conveyor belt faults are realized, and the fault diagnosis accuracy of the conveyor belt is improved. And reliable operation of the embedded equipment in a severe environment of a coal mine is ensured.

Description

Technical Field

[0001] The present invention relates to coal mine conveyor belt fault diagnosis, and in particular to a fault diagnosis method based on MobileNetV2 and TensorFlowLite. Background Art

[0002] Coal mine conveyor belts are an important part of coal mining and transportation systems, and their operating status is directly related to the efficiency and safety of the entire production system. Traditional conveyor belt fault diagnosis methods mainly rely on manual inspection and signal processing technology. These methods have many shortcomings.

[0003] 1. Limitations of manual inspection: Manual inspection requires a lot of human resources, and due to the harsh environment of coal mines, there are great safety risks in the inspection process. The frequency and time interval of manual inspection are limited, and it is difficult to find faults in time, which may cause the equipment to run in an abnormal state for a long time, resulting in serious consequences.

[0004] 2. Limitations of signal processing technology: Signal processing technology relies on prior knowledge of fault characteristics and is difficult to cope with complex and changing fault modes. Traditional signal processing methods have limited data processing and analysis capabilities and are difficult to extract effective fault information from large-scale data.

[0005] 3. The particularity of the coal mine environment: The coal mine environment is complex and changeable, and the working conditions of the conveyor belt are poor. It is easily affected by factors such as dust, moisture and high temperature, which increases the difficulty of fault diagnosis. The conveyor belt runs at a high speed, and the fault may spread in a short time after it occurs, causing large-scale equipment damage and production suspension.

[0006] With the development of industrial automation and intelligence, fault diagnosis methods based on deep learning have gradually become a research hotspot. Through automated monitoring and intelligent analysis, real-time detection and diagnosis of conveyor belt faults can be achieved. MobileNetV2, as a lightweight convolutional neural network model, has low computational complexity and high recognition accuracy, and is very suitable for deployment on resource-constrained embedded devices. TensorFlow Lite is a lightweight version of TensorFlow, designed for mobile and embedded devices, and can run efficiently on low-power devices. Summary of the invention

[0007] The purpose of this invention is to provide a coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite, which performs fault classification by training the MobileNetV2 model and converts it into the TensorFlowLite format so that it can run efficiently on embedded devices.

[0008] In order to achieve the above object, the technical solution adopted by the present invention comprises the following steps:

[0009] S1. First, data preprocessing is performed to collect image data generated during the operation of the coal mine conveyor belt and store them in .mat format files;

[0010] S2. Next, construct the MobileNetV2 model, BaseModel = MobileNet2 (weight = None, include_top = False, input_shape = (H, W, C)), where H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image;

[0011] S3, then improve the global pooling layer and adopt a weighted pooling strategy, in which the weight of each pixel is adjusted according to its position in the feature map;

[0012] S4. Then add a fully connected layer x=Dense(1024,actication='relu')(x) where Dense represents the fully connected layer, 1024 is the number of neurons in this fully connected layer, and relu represents the activation function used;

[0013] S4. Then improve the output layer, introduce dynamic attention mechanism and multi-task learning, and improve the model's ability to capture important features;

[0014] S5. Finally, in order to further improve the performance of the model, the loss function is improved, multi-task learning and attention mechanism are combined, and new loss terms are introduced to improve the robustness and generalization ability of the model.

[0015] Preferably, the data preprocessing method is to use the scipy.io library in Python to load the .mat file and perform data preprocessing, including normalization processing and division of training set and verification set. The data normalization processing is Among them, x i represents the i-th image, μ represents the mean of the data set, and σ represents the standard deviation of the data set.

[0016] Preferably, it is characterized in that the size of the improved global pooling layer for the input feature map F is H×W×C, and the output of the improved weighted average pooling layer is: Among them, G c represents the pooled output of the cth channel, w ij represents the weight of position (i, j), F ijc represents the eigenvalue of the cth channel at position (i, j). The importance of the central area is usually higher, and Gaussian distribution weighting is used to perform w ij Assignment.

[0017] Preferably, it is characterized in that the improvement step of improving the output layer in step S4 is:

[0018] S41. First, the attention mechanism is introduced. The dynamic attention mechanism enables the model to focus on important features by assigning different weights to input features, thereby improving classification performance. The improved output layer includes attention weight calculation and weighted summation.

[0019] S42, then calculate the attention weight Among them, h i represents the i-th feature, W h and b h is the weight matrix and bias term, α i is the attention weight;

[0020] S43, then perform weighted summation Among them, υ is the weighted eigenvector;

[0021] S44, then multi-task learning is carried out. Multi-task learning is introduced in the output layer. In addition to the classification task, regression tasks or other auxiliary tasks are added to improve the generalization ability of the model;

[0022] S45. The structure of the final output layer is expressed as x=Dorpout(0.5)(υ), classification_output=Dense(num_classes, activation='softmax')(x), regression_output=Dense(1, activation='leaner')(x). The final model constructed is: model=Model(input=base_model.input, output=[classification_output, regression_output]).

[0023] As a preference, the improvement of the loss function is divided into the following parts. Assuming that the output of the classification task is The true label is y, and the output of the regression task is The true value is r, the attention weight is α, and the model parameter is θ;

[0024] Part 1: Classification Error Using Weighted Cross Entropy Loss Where N is the number of samples, i is the index of the i-th sample, and w i is the weight of the ith class, y i The true label of the i-th sample, The predicted label of the i-th sample; the regression error uses Huber loss: Among them, δ is the hyperparameter of Huber loss, which controls the switching point of the loss function. The predicted value of the i-th sample, r i is the true value of the i-th sample;

[0025] Part 2: Using regularization terms and introducing L1-L2 mixed regularization to prevent overfitting and improve sparsity Among them, λ1 and λ2 are the weights of L1 and L2 regularization terms respectively, θ j is the jth parameter of the model;

[0026] Part 3: Use attention consistency loss to ensure consistent attention mechanisms between different tasks Where N is the number of dimensions in the attention mechanism, and are the attention weights of the i-th sample in the j-th dimension of task 1 and task 2 respectively;

[0027] Part 4: Using adversarial training loss and introducing adversarial sample training in, is the predicted label of the adversarial example;

[0028] Part 5: Using the entropy regularization term to introduce output entropy regularization Where C is the number of categories, is the predicted probability of the i-th sample in the j-th category;

[0029] Part 6: Using label smoothing technology to reduce the overfitting of the model to the training data in, is the parameter of smoothing labels, K is the number of categories;

[0030] Combining parts 1 to 6, the comprehensive loss function is Γ=β1Γ weighted_classification +β2Γ huber_regression +β3Γ l1l2_regularization +β4Γ attention_consistency +β5Γ adversarial

[0031] +β6Γ entropy +β7Γ label_smoothing Among them, β1 to β7 are weight coefficients used to balance various losses. For the adjustment of weight coefficients, the standard deviation of each loss item is normalized to dynamically adjust its weight coefficient so that the contribution of all loss items to the total loss is more balanced. Among them, σ(Γ i ) is the standard deviation of the i-th loss term, β i is the dynamic weight coefficient of the loss term.

[0032] Compared with the prior art, the advantages and positive effects of the present invention are:

[0033] 1. Improve the accuracy and timeliness of fault detection: Automatically extract fault features through deep learning models to achieve high-precision identification and real-time diagnosis of conveyor belt faults.

[0034] 2. Reduce manual inspection and maintenance costs: Use embedded devices for automatic monitoring and fault diagnosis to reduce reliance on manual inspections, labor costs and safety risks.

[0035] 3. Enhance the stability and safety of the system: Through real-time monitoring and early warning, timely discover and handle conveyor belt failures, prevent the spread of failures and equipment damage, and ensure the stable operation of the system.

[0036] 4. Adapt to the needs of the complex environment of coal mines: Use the lightweight MobileNetV2 model and the efficient TensorFlow Lite framework to ensure the reliable operation of embedded devices in the harsh environment of coal mines. DETAILED DESCRIPTION

[0037] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.

[0039] Embodiment, the traditional conveyor belt fault diagnosis method mainly relies on manual inspection and signal processing technology. These methods have many shortcomings. With the development of industrial automation and intelligence, the fault diagnosis method based on deep learning has gradually become a research hotspot. Therefore, the present invention proposes a coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite, which can realize high-precision identification and real-time diagnosis of conveyor belt faults, and ensure the reliable operation of embedded devices in the harsh environment of coal mines.

[0040] The first thing to consider is the collection of resource data. The image data generated during the operation of the coal mine conveyor belt is collected and stored in a .mat format file. The data preprocessing method is to use the scipy.io library in Python to load the .mat file and perform data preprocessing, including normalization processing and the division of training sets and validation sets. The data normalization processing is Among them, xi represents the i-th image, μ represents the mean of the data set, and σ represents the standard deviation of the data set.

[0041] Then we start to build the MobileNetV2 model, BaseModel = MobileNet2 (weight = None, include_top = False, input_shape = (H, W, C)), where H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image. Considering that MobileNetV2 may still require relatively more computing resources when dealing with large-scale and complex tasks, the structure of the model is improved. The first thing to consider is the improved global pooling layer. For the input feature map F, whose size is H×W×C, the output of the improved weighted average pooling layer is: Among them, G c represents the pooled output of the cth channel, w ij represents the weight of position (i, j), F ijc represents the eigenvalue of the cth channel at position (i, j). The importance of the central area is usually higher, and Gaussian distribution weighting is used to perform w ij Assignment.

[0042] Then add a fully connected layer x = Dense(1024,actication = 'relu')(x) where Dense represents the fully connected layer, 1024 is the number of neurons in this fully connected layer, and relu represents the activation function used.

[0043] The output layer is further improved by first introducing the attention mechanism. The dynamic attention mechanism enables the model to focus on important features by assigning different weights to input features, thereby improving classification performance. The improved output layer includes attention weight calculation and weighted summation; secondly, the attention weight e is calculated. i =tanh(W h h i +b h ), Among them, h i represents the i-th feature, W h and b h is the weight matrix and bias term, α i is the attention weight; then the weighted sum is performed Among them, υ is the weighted feature vector; then multi-task learning is carried out, and multi-task learning is introduced in the output layer. In addition to the classification task, regression task or other auxiliary tasks are added to improve the generalization ability of the model; finally, the output layer structure is expressed as x=Dorpout(0.5)(υ), classification_output=Dense(num_classes, activation='softmax')(x), regression_output=Dense(1, activation='leaner')(x) The final model constructed is: model=Model(input=base_model.input, output=[classification_output, regression_output]).

[0044] Finally, in order to further improve the performance of the model, the loss function is improved. Combining multi-task learning and attention mechanism, a new loss term is introduced to improve the robustness and generalization ability of the model. The improvement part is as follows: Assume that the output of the classification task is The true label is y, and the output of the regression task is The true value is r, the attention weight is α, and the model parameter is θ; Part 1: Classification error uses weighted cross entropy loss Where N is the number of samples, i is the index of the i-th sample, and w i is the weight of the ith class, y i The true label of the i-th sample, The predicted label of the i-th sample; the regression error uses Huber loss: Among them, δ is the hyperparameter of Huber loss, which controls the switching point of the loss function. The predicted value of the i-th sample, r i is the true value of the i-th sample; Part 2: Using regularization terms, introducing L1-L2 mixed regularization to prevent overfitting and improve sparsity Among them, λ1 and λ2 are the weights of L1 and L2 regularization terms respectively, θ j is the jth parameter of the model; Part 3: Using attention consistency loss to ensure that the attention mechanism between different tasks is consistent Where N is the number of dimensions in the attention mechanism, and are the attention weights of the i-th sample in the j-th dimension of task 1 and task 2 respectively; Part 4: Using adversarial training loss to introduce adversarial sample training in, is the predicted label of the adversarial sample; Part 5: Using the entropy regularization term, the regularization of the output entropy is introduced Where C is the number of categories, is the predicted probability of the i-th sample in the j-th category; Part 6: Using label smoothing technology to reduce the overfitting of the model to the training data in, is the parameter of the smoothing label, K is the number of categories; comprehensive part one to part six, the comprehensive loss function is Γ=β1Γ weighted_classification +β2Γ huber_regression +β3Γ l1l2_regularization +β4Γ attention_consistency +β5Γ adversarial +β6Γ entropy +β7Γ label_smoothing Among them, β1 to β7 are weight coefficients used to balance various losses. For the adjustment of weight coefficients, the standard deviation of each loss item is normalized to dynamically adjust its weight coefficient so that the contribution of all loss items to the total loss is more balanced. Among them, σ(Γ i ) is the standard deviation of the i-th loss term, β i is the dynamic weight coefficient of the loss term.

[0045] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite, characterized in that: The following steps are involved: S1. First, data preprocessing is performed to collect image data generated during the operation of the coal mine conveyor belt and store them in .mat format files; S2. Next, construct the MobileNetV2 model, BaseModel = MobileNet2 (weight = None, include_top = False, input_shape = (H, W, C)), where H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image; S3, then improve the global pooling layer and adopt a weighted pooling strategy, in which the weight of each pixel is adjusted according to its position in the feature map; S4. Then add a fully connected layer x=Dense(1024,actication='relu')(x) where Dense represents the fully connected layer, 1024 is the number of neurons in this fully connected layer, and relu represents the activation function used; S4. Then improve the output layer, introduce dynamic attention mechanism and multi-task learning, and improve the model's ability to capture important features; S5. Finally, in order to further improve the performance of the model, the loss function is improved, multi-task learning and attention mechanism are combined, and new loss terms are introduced to improve the robustness and generalization ability of the model.

2. The coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite according to claim 1 is characterized in that: The data preprocessing method is to use the scipy.io library in Python to load the .mat file and perform data preprocessing, including normalization processing and division of training set and verification set. The data normalization processing is Among them, x i represents the i-th image, μ represents the mean of the data set, and σ represents the standard deviation of the data set.

3. The coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite according to claim 1 is characterized in that: The improved global pooling layer has an input feature map F of size H×W×C, and the output of the improved weighted average pooling layer is: Among them, G c represents the pooled output of the cth channel, w ij represents the weight of position (i, j), F ijc represents the eigenvalue of the cth channel at position (i, j). The importance of the central area is usually higher, and Gaussian distribution weighting is used to perform w ij Assignment.

4. The coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite according to claim 1 is characterized in that: The improvement steps of step S4 to improve the output layer are: S41. First, the attention mechanism is introduced. The dynamic attention mechanism enables the model to focus on important features by assigning different weights to input features, thereby improving classification performance. The improved output layer includes attention weight calculation and weighted summation. S42, then calculate the attention weight e i =tanh(W h h i +b h ), Among them, h i represents the i-th feature, W h and b h is the weight matrix and bias term, α i is the attention weight; S43, then perform weighted summation Among them, υ is the weighted eigenvector; S44, then multi-task learning is carried out. Multi-task learning is introduced in the output layer. In addition to the classification task, regression tasks or other auxiliary tasks are added to improve the generalization ability of the model; S45. The structure of the final output layer is expressed as x=Dorpout(0.5)(υ), classification_output=Dense(num_classes, activation='softmax')(x), regression_output=Dense(1, activation='leaner')(x). The final model constructed is: model=Model(input=base_model.input, output=[classification_output, regression_output]).

5. The coal mine conveyor belt fault diagnosis method based on MobileNetV2 and TensorFlow Lite according to claim 1 is characterized in that: The improvement of the loss function is divided into the following parts. Assuming that the output of the classification task is The true label is y, and the output of the regression task is The true value is r, the attention weight is α, and the model parameter is θ; Part 1: Classification Error Using Weighted Cross Entropy Loss Where N is the number of samples, i is the index of the i-th sample, and w i is the weight of the ith class, y i The true label of the i-th sample, The predicted label of the i-th sample; the regression error uses Huber loss: Among them, δ is the hyperparameter of Huber loss, which controls the switching point of the loss function. The predicted value of the i-th sample, r i is the true value of the i-th sample; Part 2: Using regularization terms and introducing L1-L2 mixed regularization to prevent overfitting and improve sparsity Among them, λ1 and λ2 are the weights of L1 and L2 regularization terms respectively, θ j is the jth parameter of the model; Part 3: Use attention consistency loss to ensure consistent attention mechanisms between different tasks Where N is the number of dimensions in the attention mechanism, and are the attention weights of the i-th sample in the j-th dimension of task 1 and task 2 respectively; Part 4: Using adversarial training loss and introducing adversarial sample training in, is the predicted label of the adversarial example; Part 5: Using the entropy regularization term to introduce output entropy regularization Where C is the number of categories, is the predicted probability of the i-th sample in the j-th category; Part 6: Using label smoothing technology to reduce the overfitting of the model to the training data in, is the parameter of smoothing labels, K is the number of categories; Combining parts 1 to 6, the comprehensive loss function is Γ=β1Γ weighted_classification +β2Γ huber_regression +β3Γ l1l2_regularization +β4Γ attention_consistency +β5Γ adversarial +β6Γ entropy +β7Γ label_smoothing Among them, β1 to β7 are weight coefficients used to balance various losses. For the adjustment of weight coefficients, the standard deviation of each loss item is normalized to dynamically adjust its weight coefficient so that the contribution of all loss items to the total loss is more balanced. Among them, σ(Γ i ) is the standard deviation of the i-th loss term, β i is the dynamic weight coefficient of the loss term.

Images