Multi-modal information fusion carrier roller fault diagnosis method

Abstract

The invention discloses a carrier roller fault diagnosis method based on multi-modal information fusion, and particularly relates to the field of carrier roller fault diagnosis. The method comprises the following steps: collecting various modal signal data of a carrier roller, and preprocessing original one-dimensional signal data; simplifying the one-dimensional time sequence data of each signal by using classification aggregation approximation; converting each signal into a two-dimensional feature image by using a Markov field and synthesizing the two-dimensional feature image into a three-channel image; a carrier roller fault diagnosis model based on a Swin Transform network architecture is established; the Swin Transform network architecture is trained and verified, if the network achieves the optimal effect, the next step is carried out, otherwise, the previous step is returned, and initial parameters are modified to continue training; and inputting the test set into the optimal Swin Transform network architecture to carry out carrier roller fault diagnosis, and finally obtaining a carrier roller fault classification result. According to the technical scheme, the problems that existing carrier roller fault diagnosis is based on single-mode information, and information is insufficient and inaccurate are solved, and a foundation is laid for follow-up control and maintenance implementation.

Description

Technical Field

[0001] The invention relates to the field of roller fault diagnosis, and in particular to a roller fault diagnosis method based on multi-modal information fusion. Background Art

[0002] Belt conveyor is a typical rotating machine, mainly composed of conveyor belt, roller, roller, frame, drive device and tensioning device. It has the advantages of large transport capacity, long-distance transport and continuous transport, and is widely used in logistics, energy and resource development and other fields. The drive device is the power source of the belt conveyor, which largely determines the operating efficiency of the belt conveyor. The roller is the main component of the drive device failure, which mainly plays a bearing and guiding role. The number of rollers can reach thousands of sets, and their weight accounts for about 35% of the belt conveyor, and the value accounts for about 30%-40% of the belt conveyor. Unexpected downtime caused by belt conveyor failure will not only cause huge economic losses, but also may cause malignant secondary accidents such as fire and explosion. In the belt conveyor fire accident, high temperature fire caused by roller failure is one of the main reasons. Wear, bending and deformation of the roller shell, and bearing failure can easily cause the roller to stall. The continuous friction between the conveyor belt and the roller surface accumulates a large amount of heat. After the vehicle stops, the static contact between the conveyor belt and the roller generates continuous high temperature, which causes fire when it reaches the ignition point, and then causes an accident.

[0003] At present, the research on roller fault diagnosis is mainly based on single modal information such as sound, vibration, and image, ignoring the different representation forms of each modal information for the same fault type, and lacking an effective mechanism for the fusion and utilization of multi-modal information. Different modalities have different ways of expression and different perspectives on things, so there are some cross-complementary phenomena of features, and there may even be multiple different information interactions between modalities. If multi-modal information can be processed reasonably, rich feature information can be obtained. At the same time, the single-modal fault diagnosis method has the problems of insufficient information and inaccuracy. Summary of the invention

[0004] The present invention aims to provide a roller fault diagnosis method based on multi-modal information fusion, which solves the problem that the existing roller fault diagnosis is based on single modal information and has insufficient and inaccurate information.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: a roller fault diagnosis method based on multi-modal information fusion, comprising the following steps:

[0006] S1. Collect multiple modal signal data during the operation of the roller, and pre-process the collected original one-dimensional signal data;

[0007] S2, use classification aggregation to approximate and simplify the one-dimensional time series data of each signal;

[0008] S3, converting each signal after the aggregation process into a two-dimensional feature image using a Markov field;

[0009] S4, synthesize the two-dimensional feature images of all signals into three-channel images, and divide them into training set, verification set and test set according to 6:3:1;

[0010] S5. Establish a roller fault diagnosis model based on the Swin Transformer network architecture and initialize the parameters;

[0011] S6, train and verify the Swin Transformer network architecture based on the training set and the validation set, and optimize the network parameters using the optimization algorithm. If the network achieves the optimal effect, proceed to step S7, otherwise proceed to step S5, modify the initial parameters, continue training, and save the optimal weights;

[0012] S7. Input the test set into the optimal Swin Transformer network architecture in step S6 to perform roller fault diagnosis, and finally obtain the roller fault classification result and test accuracy.

[0013] Furthermore, the multiple modal signal data include temperature signals, sound signals and vibration signals.

[0014] Furthermore, the preprocessing method of step S1 is: segment to obtain each group of sequence data with a length of m, that is, the temperature time series data T = {t1, t2, ..., t i ,…,t m}, audio time series data S = {s1, s2, ..., s i ,…,s m}, vibration time series data V = {v1, v2, ..., v i , …, v m}.

[0015] Furthermore, the sequence data of each signal in step S1 is approximated by segmented aggregation to reduce the data dimension, that is, a time series data with a length of n is used To represent the original time series data X with a length of m = {x1, x2, ..., x i , …, x m}, where n≤m, the specific calculation formula is as follows:

[0016]

[0017] Where: k is the compression ratio, k must be an integer.

[0018] Furthermore, the Markov field used in step S3 is a special case of the Markov random field, assuming that the value of a certain position in the random field is only related to the values ​​of the positions adjacent to it, and the calculation method is as follows:

[0019] The time series X = {x1, x2, ..., x i , …, x n}Divide the value range into Q quantile units, using quantile q j (j∈[1,Q]) quantizes each value of the time series, converting each value x i Mapped to the corresponding quantile q i , define the matrix W Q×Q , where w i，j represents the probability that the signal point in unit j is followed by the element in unit i, that is, the migration probability is calculated in the first Markov chain manner along the time step:

[0020] w i，j =P(x t ∈q i |x t-1 ∈q j )

[0021] Matrix W Q×Q That is, the Markov state transfer matrix. The Markov field expands the Markov state transfer matrix by arranging each migration probability in time order. The migration probability matrix M n×n , where m i，j is the probability of an element in cell i migrating to cell j:

[0022] m i，j =P(q i →q j )

[0023] Matrix M n×n That is the Markov field characteristic graph:

[0024]

[0025] Furthermore, in step S4, each set of time series data is fused through spatial dimension data, that is, the encoded single-channel Markov field feature map is superimposed on the depth dimension to obtain a multi-channel Markov field feature map, and the training set, validation set and test set are divided into 6:3:1.

[0026] Furthermore, the roller fault diagnosis model of step S5 includes an image block segmentation layer, a cascading module, a normalization layer, a global pooling layer and a fully connected layer. The image block segmentation layer divides the multi-channel Markov field feature map into image blocks of 4×4 adjacent pixels each and flattens it in the channel direction of the input image. The cascading module constructs feature maps of different sizes through four stages. The number of image block sequences through each stage decreases as the network deepens. Finally, the roller fault diagnosis result is output through normalization, global pooling and fully connected layers.

[0027] Furthermore, in step S6, the loss function is used to optimize the network parameters by back-propagating the errors generated by the predicted samples and the true sample labels, and the cross entropy is selected as the loss function:

[0028]

[0029] Where: q(x i ) is the predicted probability distribution, p(x i ) is the true probability distribution of the sample,

[0030] The AdamW optimizer is used in back propagation to train and optimize the network parameters, which is different from the weight decay in the Adma optimizer before calculating the gradient.

[0031] Compared with the prior art, this solution has the following beneficial effects:

[0032] 1. This solution collects multiple modal signal data during the operation of the roller, and diagnoses the health status of the roller by fusing multiple modal signal data, overcoming the problems of insufficient information and inaccuracy in the fault diagnosis method based on single modal information.

[0033] 2. This scheme uses the advantage of Markov Transition Field (MTF) to consider the time correlation of signals, encodes the original one-dimensional signal data, obtains the data correlation in different time intervals according to the transition probability between the data, and generates a two-dimensional feature image with time correlation. This encoding conversion process is essentially a dimensionality increase process, which adds the relationship representation between data values ​​at different times compared to the original one-dimensional signal data. At the same time, the original data is simplified using piecewise aggregate approximation (PAA) to control the sample length of the signal, and prevent the large-size feature map obtained after the MTF encoding conversion from increasing the calculation and parameter amount of the subsequent network.

[0034] 3. This solution converts multiple modal signal data into a synthesized multi-channel MTF image to achieve information fusion of the data layer while retaining their respective time correlations. Then, the Swin Transformer network is used to train the multi-channel MTF graph containing multiple modal signal data, wherein Swin Transformer is an improvement on Vision Transformer, adopts a hierarchical construction method, uses different multiples of downsampling to construct feature maps, and uses the window multi-head attention mechanism (Windows Multi-Head Self-Attention, W-MSA) to perform attention calculations in each window. And the sliding window multi-head attention mechanism (Shifted Windows Multi-Head Self-Attention, SW-MSA) module is used to transfer information between adjacent windows. The amount of calculation and the amount of parameters can be effectively reduced while ensuring the feature extraction capability. Finally, a global pooling layer and a fully connected layer are added after the Swin Transformer structure to obtain the final fault diagnosis result output. Therefore, the present invention can accurately and effectively classify and identify faults occurring during the operation of the roller. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a general block diagram of a roller fault diagnosis method of multi-modal information fusion of the present invention;

[0036] Figure 2 It is a flow chart of a roller fault diagnosis method of multi-modal information fusion of the present invention;

[0037] Figure 3 It is a diagram of the Swin Transformer network structure used in this embodiment. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below through specific embodiments:

[0039] Example

[0040] like Figures 1 to 3 As shown, a roller fault diagnosis method based on multi-modal information fusion includes the following steps:

[0041] S1. The temperature, sound, vibration and other modal signal data of the roller during operation are collected by sensors, and the collected raw data are preprocessed. The preprocessing method is as follows: each group of sequence data with a length of m is divided into temperature time series data T = {t1, t2, ..., t i ,…,t m}, audio time series data S = {s1, s2, ..., s i ,…,sm}, vibration time series data V = {v1, v2, ..., v i , …, v m}.

[0042] S2: Use segmented aggregation approximation to reduce the dimension of the sequence data of each signal in step S1, that is, use a time series data with a length of n To represent the original time series data X with a length of m = {x1, x2, ..., x i , …, x m}, where n≤m, the specific calculation formula is as follows:

[0043]

[0044] Where: k is the compression ratio, k must be an integer.

[0045] S3, the temperature time series data after the polymerization process in step S2 Sound timing data Vibration time series data The Markov field is used to convert time series data into two-dimensional feature images. The Markov field is a method for converting time series data into spatial image data. This method expands the Markov state transition matrix and fully retains the discrete time domain dynamic information by sequentially expressing the state transition matrix. It captures the time dependence and dynamic changes in the time series in the form of images, revealing new patterns or structural features that are not obvious or difficult to detect in the original data of various types, thereby more effectively capturing complex patterns and anomalies.

[0046] S4. The Markov field used in step S3 is a special case of the Markov random field, which assumes that the value of a certain position in the random field is only related to the values ​​of the positions adjacent to it. The Markov field states the value of the time series, and then calculates the transformation probability of the time series, indicating the relationship between the data at any two time points in the time series data, thereby capturing the distribution change information. i , …, x n} is divided into Q quantile units according to its value range, and the quantile q j (j∈[1,Q]) quantizes each value of the time series, converting each value x i Mapped to the corresponding quantile q i . Define the matrix W Q×Q , where w i，j represents the probability that the signal point in unit j is followed by the element in unit i, that is, the migration probability is calculated in the first Markov chain manner along the time step:

[0047] w i，j =P(xt ∈q i |x t-1 ∈q j )

[0048] Matrix W Q×Q This is the Markov state transfer matrix, which ignores the distribution of the time series X and the time step t i The Markov field arranges each migration probability in time sequence to expand the Markov state transfer matrix. The migration probability matrix M n×n , where m i，j is the probability of an element in cell i migrating to cell j:

[0049] m i，j =P(q i →q j )

[0050] Matrix M n×n That is the Markov field characteristic graph:

[0051]

[0052] Based on the single-channel Markov field feature map obtained by encoding the temperature, sound, and vibration time series data in step S4, this set of time series data is fused with spatial dimension data to obtain a two-dimensional image with three channels, that is, the three encoded single-channel Markov field feature maps are superimposed on the depth dimension, which is essentially a three-dimensional array, and the size of the obtained multi-channel Markov field feature map is n×n×3. After the groups of data collected and processed in step S1 are converted into multi-channel Markov field feature maps through the above steps, they are divided into training set, verification set and test set according to the ratio of 6:3:1.

[0053] S5. Construct a roller fault diagnosis model based on the Swin Transformer network architecture, including an image block segmentation layer, a cascade module, a normalization layer, a global pooling layer, and a fully connected layer. The image block segmentation layer divides the multi-channel Markov field feature map obtained in step S4 into image blocks of 4×4 adjacent pixels each, and flattens it in the channel direction of the input image. The cascade module constructs feature maps of different sizes through four stages. The number of image block sequences in each stage decreases as the network deepens, thereby achieving hierarchical representation. Finally, the roller fault diagnosis results are output through normalization, global pooling, and fully connected layers.

[0054] S6. Use the training set and validation set obtained in step S4 to train and validate the network model constructed in step S5. Use the loss function to back-propagate the error generated by the predicted sample and the real sample label to optimize the network parameters, and select cross entropy as the loss function:

[0055]

[0056] Where: q(x i ) is the predicted probability distribution, p(x i ) is the true probability distribution of the sample.

[0057] The AdamW optimizer is used in back propagation to train and optimize network parameters. Unlike the weight decay in the Adma optimizer before calculating the gradient, AdamW applies weight decay after the gradient calculation is completed. Therefore, using AdamW usually has a faster convergence speed and better generalization.

[0058] S7. Input the test set obtained in step S4 into the optimal model trained in step S6 to perform roller fault diagnosis, and finally obtain the roller fault classification result and test accuracy.

[0059] The above are only embodiments of the present invention, and the common knowledge such as the known specific structures and / or characteristics in the scheme are not described in detail here. It should be pointed out that for those skilled in the art, several deformations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A roller fault diagnosis method based on multi-modal information fusion, characterized in that: The steps include: S1. Collect multiple modal signal data during the operation of the roller, and pre-process the collected original one-dimensional signal data; S2, use classification aggregation to approximate and simplify the one-dimensional time series data of each signal; S3, converting each signal after the aggregation process into a two-dimensional feature image using a Markov field; S4, synthesize the two-dimensional feature images of all signals into three-channel images, and divide them into training set, verification set and test set according to 6:3:1; S5. Establish a roller fault diagnosis model based on the Swin Transformer network architecture and initialize the parameters; S6, train and verify the Swin Transformer network architecture based on the training set and the validation set, and optimize the network parameters using the optimization algorithm. If the network achieves the optimal effect, proceed to step S7, otherwise proceed to step S5, modify the initial parameters, continue training, and save the optimal weights; S7. Input the test set into the optimal Swin Transformer network architecture in step S6 to perform roller fault diagnosis, and finally obtain the roller fault classification result and test accuracy.

2. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: The various modal signal data include temperature signals, sound signals and vibration signals.

3. The roller fault diagnosis method based on multi-modal information fusion according to claim 2 is characterized in that: The preprocessing method of step S1 is: segment to obtain each group of sequence data with a length of m, that is, temperature time series data T = {t1, t2, ..., t i ,…,t m }, audio time series data S = {s1, s2, ..., s i ,…,s m }, vibration time series data V = {v1, v2, ..., v i , …, v m }.

4. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: The sequence data of each signal in step S1 is approximated by segmented aggregation to reduce the data dimension, that is, a time series data with a length of n is used To represent the original time series data X with a length of m = {x1, x2, ..., x i , …, x m }, where n≤m, the specific calculation formula is as follows: Where: k is the compression ratio, k must be an integer.

5. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: The Markov field used in step S3 is a special case of the Markov random field, assuming that the value of a position in the random field is only related to the values ​​of the positions adjacent to it. The calculation method is as follows: The time series X = {x1, x2, ..., x i , …, x n }Divide the value range into Q quantile units, using quantile q j (j∈[1,Q]) quantizes each value of the time series, converting each value x i Mapped to the corresponding quantile q i , define the matrix W Q×Q , where w i，j represents the probability that the signal point in unit j is followed by the element in unit i, that is, the migration probability is calculated in the first Markov chain manner along the time step: w i，j =P(x t ∈q i |x t-1 ∈q j ) Matrix W Q×Q That is, the Markov state transfer matrix. The Markov field expands the Markov state transfer matrix by arranging each migration probability in time order. The migration probability matrix M n×n , where m i，j is the probability of an element in cell i migrating to cell j: m i,j =P(q i →q j ) Matrix M n×n That is the Markov field characteristic graph:

6. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: In step S4, each set of time series data is fused through spatial dimension data, that is, the encoded single-channel Markov field feature map is superimposed on the depth dimension to obtain a multi-channel Markov field feature map, and the training set, validation set and test set are divided into 6:3:

1.

7. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: The roller fault diagnosis model of step S5 includes an image block segmentation layer, a stacking module, a normalization layer, a global pooling layer and a fully connected layer. The image block segmentation layer divides the multi-channel Markov field feature map into image blocks of 4×4 adjacent pixels each and flattens it in the channel direction of the input image. The stacking module constructs feature maps of different sizes through four stages. The number of image block sequences through each stage decreases as the network deepens. Finally, the roller fault diagnosis result is output through normalization, global pooling and fully connected layers.

8. The roller fault diagnosis method based on multi-modal information fusion according to claim 1 is characterized in that: In step S6, the loss function is used to optimize the network parameters by back-propagating the errors between the predicted samples and the true sample labels, and the cross entropy is selected as the loss function: Where: q(x i ) is the predicted probability distribution, p(x i ) is the true probability distribution of the sample, The AdamW optimizer is used in back propagation to train and optimize the network parameters, which is different from the weight decay in the Adma optimizer before calculating the gradient.

Images