A multi-modal information fusion roller fault diagnosis method

Abstract

The patent discloses a kind of multi-modal information fusion's idler fault diagnosis method, specifically related to idler fault diagnosis field.The steps are as follows: collecting the multi-modal signal data of idler, the original one-dimensional signal data is preprocessed;One-dimensional time series data of each signal is simplified using classification aggregation;Each signal is converted into two-dimensional feature image using Markov field and is combined into three-channel image;Establish the idler fault diagnosis model based on Swin Transformer network architecture;Swin Transformer network architecture is trained and verified, if network reaches optimal effect, then next step is carried out, otherwise return to previous step, and modify initial parameter and continue training;Test set is input into optimal Swin Transformer network architecture to carry out idler fault diagnosis, and finally obtain idler fault classification result.The technical scheme of the present application solves the problem of insufficient and inaccurate information in existing idler fault diagnosis based on single modal information, and lays a foundation for subsequent implementation control and maintenance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of idler fault diagnosis, in particular to a kind of multi-modal information fusion's idler fault diagnosis method. BACKGROUND

[0002] Belt conveyor is a typical rotating machinery, mainly by conveying belt, drum, idler, frame, driving device and tensioning device etc., with large capacity, long distance transportation and continuous transportation etc., is widely used in logistics transportation, energy and resource development and many other fields.Driving device is the power source of belt conveyor, largely determines the running efficiency of belt conveyor.Idler is the main component of driving device, mainly plays the role of bearing and guiding, its quantity can reach thousands of groups, its weight accounts for about 35% of belt conveyor, and its value accounts for about 30%-40% of belt conveyor.In belt conveyor fire accident, idler fault causes high temperature fire, which is one of the main reasons.Idler cylinder skin wear, bending deformation, bearing failure can easily cause idler locked-rotor, and the continuous friction between conveying belt and idler surface can accumulate a lot of heat.After parking, the continuous high temperature generated by the static contact between conveying belt and idler can reach the ignition point and cause fire, and further cause accidents.

[0003] At present, the research on idler fault diagnosis is mainly based on sound, vibration, image and other single modal information, ignores the different representation forms of various modal information for the same type of fault, and lacks effective mechanism for multi-modal information fusion.Different modalities have different ways of expression and different perspectives on things, so there is some characteristic cross-complementary phenomenon, and there may be multiple information interactions between modalities.If multi-modal information can be reasonably processed, rich feature information can be obtained.Meanwhile, the single modal fault diagnosis method has the problems of insufficient information and inaccuracy. SUMMARY

[0004] The present application aims to provide a kind of multi-modal information fusion's idler fault diagnosis method, solve the existing idler fault diagnosis based on single modal information, there is the problem of insufficient information and inaccuracy.

[0005] In order to achieve the above purpose, the technical scheme of the present application is as follows: a kind of multi-modal information fusion's idler fault diagnosis method, comprising the following steps:

[0006] S1, collecting various modal signal data in the running process of idler, and pre-processing the collected original one-dimensional signal data;

[0007] S2, using classification aggregation to simplify one-dimensional time series data of each signal;

[0008] S3, converting each signal processed by aggregation into a two-dimensional feature image using Markov field.

[0009] S4, synthesize the two-dimensional feature images of all signals into three-channel images, and divide them into training set, validation set and test set according to 6:3:1;

[0010] S5. Establish a roller fault diagnosis model based on the Swin Transformer network architecture and initialize the parameters;

[0011] S6. Train and verify the Swin Transformer network architecture based on the training set and the validation set, and optimize the network parameters using the optimization algorithm. If the network reaches the optimal effect, proceed to step S7; otherwise, proceed to step S5, modify the initial parameters, continue training, and save the optimal weights;

[0012] S7. Input the test set into the optimal Swin Transformer network architecture in step S6 to perform roller fault diagnosis, and finally obtain the roller fault classification result and test accuracy.

[0013] Furthermore, the multiple modal signal data include temperature signals, sound signals and vibration signals.

[0014] Furthermore, the preprocessing method of step S1 is: segment to obtain each group of sequence data with a length of m, that is, the temperature time series data T = {t1, t2, ..., t i ,…,t m}、Sound time series data S={s1,s2,…,s i ,…,s m}、Vibration time series data V={v1,v2,…,v i ,…,v m}.

[0015] Furthermore, the sequence data of each signal in step S1 is approximated by segmented aggregation to reduce the data dimension, that is, a time series data with a length of n is used To represent the original time series data X with a length of m = {x1, x2, ..., x i ,…,x m}, where n≤m, the specific calculation formula is as follows:

[0016]

[0017] Where: k is the compression ratio, k must be an integer.

[0018] Furthermore, the Markov field used in step S3 is a special case of the Markov random field, assuming that the value of a certain position in the random field is only related to the values ​​of the positions adjacent to it. The calculation method is as follows:

[0019] The time sequence X={x1, x2, …, x i , …, x n} is divided into Q quantile units according to the value range, each value of the time sequence is quantized by quantile q j (j∈[1, Q]), each value x i is mapped to the corresponding quantile q i , and a matrix W Q×Q is defined, wherein w i，j represents the probability that the signal point in unit j is followed by the element in unit i, that is, the transition probability is calculated in the first Markov chain along the time step:

[0020] w i，j =P(x t ∈q i |x t-1 ∈q j )

[0021] The matrix W Q×Q is a Markov state transition matrix, and the Markov field extends the Markov state transition matrix by arranging each transition probability in time sequence, and the transition probability matrix M n×n , wherein m i，j is the probability that the element in unit i moves to unit j:

[0022] m i，j =P(q i →q j )

[0023] The matrix M n×n is a Markov field feature map:

[0024]

[0025] Further, in step S4, each group of time sequence data is fused in the spatial dimension, that is, the single-channel Markov field feature map formed by encoding is superimposed in the depth dimension to obtain a multi-channel Markov field feature map, and the training set, the validation set and the test set are divided according to 6:3:1.

[0026] Further, the roller fault diagnosis model of step S5 includes an image block segmentation layer, a layer stacking module, a normalization layer, a global pooling layer and a full connection layer, the image block segmentation layer divides the multi-channel Markov field feature map into an image block of every 4x4 adjacent pixels, and flattens in the channel direction of the input picture, the layer stacking module constructs feature maps of different sizes through four stages, the number of image block sequences in each stage decreases as the network deepens, and finally the roller fault diagnosis result is output through the normalization, global pooling and full connection layers.

[0027] Further, in step S6, the network parameters are optimized by using a loss function to perform back propagation on the error generated by the predicted sample and the real sample label, and cross entropy is selected as the loss function:

[0028]

[0029] In the formula: q(x i ) is a predicted probability distribution, p(x i ) is a real probability distribution of a sample,

[0030] In the back propagation, an AdamW optimizer is used to train and optimize the network parameters, which is different from the weight decay in the Adma optimizer before calculating the gradient.

[0031] Compared with the prior art, the beneficial effects of the present scheme are:

[0032] 1. The present scheme collects various modal signal data in the running process of the roller, and diagnoses the health status of the roller through fusion of various modal signal data, thereby overcoming the problems of insufficient information and inaccuracy in the fault diagnosis method based on single modal information.

[0033] 2. The present scheme uses the advantage of Markov Transition Field (MTF) to consider the time correlation of signals, encodes the original one-dimensional signal data, obtains the data correlation in different time intervals according to the transition probability between data, and generates a two-dimensional feature image with time correlation. The encoding and conversion process is essentially a dimensionality increasing process, which increases the representation of the relationship between data values at different times compared to the original one-dimensional signal data. At the same time, the original data is simplified using Piecewise Aggregate Approximation (PAA) to control the sample length of the signal and prevent the large size feature map obtained after MTF encoding and conversion from increasing the calculation and parameter amount of the subsequent network.

[0034] 3、The scheme converts and synthesizes multiple modal signal data into a multi-channel MTF image, realizes information fusion at the data layer, and at the same time retains the respective correlation with time. Then, a Swin Transformer network is used to train the multi-channel MTF image containing multiple modal signal data, wherein the Swin Transformer is an improvement on the Vision Transformer, adopts a hierarchical construction method, uses different multiples of downsampling to construct feature maps, and uses a window multi-head attention mechanism (Windows Multi-Head Self-Attention, W-MSA) in each window to perform attention calculation. And use the Shifted Windows Multi-Head Self-Attention (SW-MSA) module to pass information between adjacent windows. The calculation amount and parameter amount can be effectively reduced under the premise of ensuring the feature extraction capability. Finally, a global pooling layer and a fully connected layer are added after the Swin Transformer structure to obtain the final fault diagnosis result output. Therefore, the present application can accurately and effectively classify and identify the faults occurring in the running process of the roller. BRIEF DESCRIPTION OF DRAWINGS

[0035] Fig. 1 is a general block diagram of a multi-modal information fusion roller fault diagnosis method of the present application;

[0036] Fig. 2 is a flowchart of a multi-modal information fusion roller fault diagnosis method of the present application;

[0037] Fig. 3 is a Swin Transformer network structure diagram used in the present embodiment. DETAILED DESCRIPTION

[0038] The present application will be further described in detail through specific embodiments:

[0039] EMBODIMENT

[0040] As shown in Figs. 1 to 3 , a multi-modal information fusion roller fault diagnosis method comprises the following steps:

[0041] S1, collect temperature, sound, vibration and other multi-modal signal data in the running process of the roller through a sensor, and pre-process the collected raw data. The pre-processing method is to divide each group of sequence data with a length of m into temperature time series data T={t1, t2, …, t i , …, t m}, sound time series data S={s1, s2, …, s i , …, sm}、Vibration time series data V={v1,v2,…,v i ,…,v m}.

[0042] S2, the sequence data of each signal in step S1 is approximated by segmented aggregation to reduce the data dimension, that is, a time series data with a length of n is used To represent the original time series data X with a length of m = {x1, x2, ..., x i ,…,x m}, where n≤m, the specific calculation formula is as follows:

[0043]

[0044] Where: k is the compression ratio, k must be an integer.

[0045] S3, the temperature time series data after the aggregation process in step S2 Sound timing data Vibration time series data Each is converted into a two-dimensional feature image using a Markov field. Markov fields are a method for converting time series data into spatial image data. This method extends the Markov state transition matrix and, by expressing it sequentially, fully preserves the dynamic information in the discrete time domain. By capturing the temporal dependencies and dynamic changes in time series in the form of images, they reveal new patterns or structural features that are less obvious or difficult to detect in the original data, thereby more effectively capturing complex patterns and anomalies.

[0046] S4. The Markov field used in step S3 is a special case of the Markov random field. Its premise is to assume that the value of a certain position in the random field is only related to the values ​​of the positions adjacent to it. The Markov field states the value of the time series and then calculates the transformation probability of the time series, indicating the relationship between the data at any two time points in the time series data, thereby capturing the change information of the distribution. i ,…,x n} is divided into Q quantile units according to its value range, and quantile q is used j (j∈[1,Q]) quantizes each value of the time series, and converts each value x i Mapped to the corresponding quantile q i . Define the matrix W Q×Q , where w i，j represents the probability that the signal point in unit j is followed by the element in unit i, that is, the migration probability is calculated in the first Markov chain manner along the time step:

[0047] w i，j =P(xt ∈q i |x t-1 ∈q j )

[0048] Matrix W Q×Q is the Markov state transition matrix, which ignores the time-dependent relationship between the distribution of time series X and time step t i . The Markov field extends the Markov state transition matrix by arranging each transition probability in time order. The transition probability matrix M n×n , where m i，j is the probability of an element in unit i migrating to unit j:

[0049] m i，j = P(q i → q j )

[0050] Matrix M n×n is the Markov field feature map:

[0051]

[0052] Based on step S4, the single-channel Markov field feature map obtained by encoding the temperature, sound, and vibration time series data, and after spatial dimension data fusion, a two-dimensional image with three channels is obtained by conversion, that is, the three single-channel Markov field feature maps formed by encoding are superimposed in the depth dimension, which is essentially a three-dimensional array, and the obtained multi-channel Markov field feature map has a size of n x n x 3. After the groups of data collected and processed in step S1 are converted into multi-channel Markov field feature maps through the above steps, the training set, validation set, and test set are divided according to the proportion of 6:3:1.

[0053] S5, the roller fault diagnosis model based on the Swin Transformer network architecture includes an image block segmentation layer, a layer stacking module, a normalization layer, a global pooling layer, and a full connection layer. The image block segmentation layer divides the multi-channel Markov field feature map obtained in step S4 into image blocks with every 4 x 4 adjacent pixels as one image block, and flattens in the channel direction of the input picture. The layer stacking module constructs feature maps of different sizes through four stages, and the number of image block sequences in each stage decreases as the network deepens, thereby realizing hierarchical representation. Finally, the roller fault diagnosis result is output through the normalization, global pooling, and full connection layers.

[0054] S6, the training set and validation set obtained in step S4 are used for training and validation of the network model constructed in step S5. The loss function is used to propagate the error generated by the predicted sample and the real sample label in the reverse direction, thereby optimizing the network parameters, and cross-entropy is selected as the loss function:

[0055]

[0056] where q(x i ) is the predicted probability distribution, and p(x i ) is the true probability distribution of the sample.

[0057] The AdamW optimizer is used in back propagation to optimize the training of network parameters. Unlike the weight decay in the Adma optimizer, the AdamW applies weight decay after the gradient is calculated. Therefore, using AdamW usually has faster convergence speed and better generalization.

[0058] S7, input the test set obtained in step S4 into the optimal model trained in step S6 for roller fault diagnosis, and finally obtain the roller fault classification result and the test accuracy.

[0059] The above is only an embodiment of the present application, and the well-known specific structures and / or characteristics in the scheme are not described in detail. It should be pointed out that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should also be regarded as the protection scope of the present application, and these will not affect the effect and practicality of the patent. The protection scope claimed in the present application should be subject to the content of its claims, and the specific implementation mode and the like recorded in the specification can be used to explain the content of the claims.

Claims

1. A roller fault diagnosis method based on multimodal information fusion, characterized by: The method comprises the following steps: S1, collecting various modal signal data in the running process of the roller, and preprocessing the collected original one-dimensional signal data; S2, using classification aggregation to simplify the one-dimensional time series data of each signal; S3, converting each signal after aggregation processing into a two-dimensional feature image using a Markov field; S4, combining the two-dimensional feature images of all signals into a three-channel image, and dividing the training set, validation set and test set according to the ratio of 6:3:1; S5, establishing a roller fault diagnosis model based on the Swin Transformer network architecture, and initializing the parameters; S6, training and verifying the Swin Transformer network architecture based on the training set and the validation set, optimizing the network parameters using an optimization algorithm, if the network reaches the optimal effect, then performing step S7, otherwise performing step S5, and modifying the initial parameters to continue training and saving the best weight; S7, inputting the test set into the optimal Swin Transformer network architecture in step S6 for roller fault diagnosis, and finally obtaining the roller fault classification result and the test accuracy; The Markov field used in step S3 is a special case of Markov random field, provided that the value of a certain position in the random field is only related to the values of adjacent positions, and the calculation method is as follows: The time series sequence is divided into sub-ranges according to the value range Q Each sub-range is assigned a quantile Each value of the time series is quantized to a quantile The quantile is mapped to a corresponding quantile A matrix is defined, where represents the probability that a signal point in cell j is followed by an element in cell i , i.e. the transition probabilities are calculated in a first Markov chain fashion along the time steps matrix is the Markov state transition matrix, the Markov field extends the Markov state transition matrix by arranging each transition probability in a temporal order, the transition probability matrix where is the element i in the cell j transitions to the cell matrix i.e. the Markov field feature map: ； In step S4, each group of time series data is fused in the spatial dimension, that is, the single-channel Markov field feature map formed by encoding is stacked in the depth dimension to obtain a multi-channel Markov field feature map, and the training set, validation set and test set are divided according to the ratio of 6:3:

1.

2. The multi-modal information fusion method for the roller fault diagnosis according to claim 1, characterized in that: The various modal signal data include temperature signals, sound signals and vibration signals.

3. The multi-modal information fusion method for the roller fault diagnosis according to claim 2, characterized in that: The preprocessing method of step S1 is: dividing each group of sequence data with length m, i.e. temperature time series data , sound time series data , vibration time series data .

4. The multi-modal information fusion method for diagnosing the idler roller fault according to claim 1, characterized in that: The sequence data of each signal in step S1 is approximated by segmented aggregation to reduce the data dimension, that is, a length of n Time series data To indicate the length m The original time series data ,in , the specific calculation formula is as follows: In the formulae: k is the compression rate, , k must be an integer.

5. The multi-modal information fusion method for the roller fault diagnosis according to claim 1, characterized in that: The roller fault diagnosis model of step S5 includes an image block segmentation layer, a layer stacking module, a normalization layer, a global pooling layer and a full connection layer, the image block segmentation layer divides the multi-channel Markov field feature map into an image block of 4x4 adjacent pixels, and flattens the input picture in the channel direction, the layer stacking module constructs feature maps of different sizes through four stages, the number of image blocks in each stage decreases as the network deepens, and finally the roller fault diagnosis result is output through the normalization, global pooling and full connection layers.

6. The multi-modal information fusion idler fault diagnosis method according to claim 1, characterized in that: In step S6, the loss function is used to optimize the network parameters by backpropagating the error generated by the predicted sample and the real sample label, and cross-entropy is selected as the loss function: wherein: is the predicted probability distribution, is the sample true probability distribution, In the backpropagation, the AdamW optimizer is used to train and optimize the network parameters, which is different from the weight decay in the Adma optimizer before calculating the gradient.

Images