Abnormal sound detection method for fault diagnosis of belt conveyor

Abstract

The invention discloses an abnormal sound detection method for fault diagnosis of a belt conveyor. The abnormal sound detection method comprises the following steps: constructing a training set; the training set comprises normal sound and abnormal sound; and training a preset belt conveyor fault diagnosis sound anomaly detection network by using the training set. The preset sound anomaly detection method for fault diagnosis of the belt conveyor mainly comprises a logarithmic Mel spectrogram module, a spectrum-time feature fusion module and a flow-based self-supervision estimation module. Effective features of the operation sound of the belt conveyor in the noisy environment are extracted, fault diagnosis is carried out, and whether a fault occurs or not and the fault type are judged. The logarithmic Mel spectrogram module and the spectral time feature fusion module are fused on the stream-based self-supervised density estimation module, so that multi-scale feature fusion is realized, extracted features are enriched, and an excellent abnormal sound detection effect of belt conveyor fault diagnosis is obtained.

Description

Technical Field

[0001] The invention belongs to the field of abnormal sound detection and relates to an abnormal sound detection method for belt conveyor fault diagnosis. Background Art

[0002] With the continuous development of science and technology and the emergence of labor shortage, the large-scale use of industrial equipment to replace manual production has become more and more popular. The fault detection technology of industrial equipment using sound signals has the advantages of non-contact measurement, simple equipment, and easy acquisition of detection signals, and has great development potential. Abnormal sound detection (ASD) is the task of identifying whether a given sound is normal or abnormal. Since there is less abnormal sound data and there are many types, normal sounds are usually used for training. Then, they calculate the error between the observed sound based on the degree of fit between the sound and the learning model. If the error exceeds the preset threshold, the sound is identified as abnormal.

[0003] Traditional ASD systems use autoencoders (AEs), such as interpolation deep neural networks (IDNNs) and ID conditional autoencoders. They learn the features of normal sounds by minimizing the reconstruction error and using the reconstruction error as the score for detecting anomalies. However, the effectiveness of such models may be limited because the training process does not involve abnormal sounds. Compared with unsupervised classification methods based on autoencoders (AEs), such as interpolation deep neural networks (IDNNs) [4] and ID conditional autoencoders, methods using self-supervised classification perform better. Self-supervised classification methods can automatically generate labels using abundant data and have higher detection accuracy. However, this method is not always stable, and its performance varies even for machines of the same type. To alleviate this problem, researchers proposed a flow-based self-supervised density estimation (i.e., GlowAff), using a normalized flow such as a generative flow (Glow) or a masked autoregressive flow (MAF). The detection performance of a machine ID is improved by introducing an auxiliary task to distinguish the sound data of the machine ID (target data) from the sound data of other machine IDs with the same machine type (outlier data). Therefore, this approach requires using different training models for different machine IDs of each machine type, which is undesirable for general applications.

[0004] The log-Mel spectrogram is designed based on human auditory perception and uses a Mel filter bank to capture information of various frequencies, so it is widely used in ASD. However, it may filter out high-frequency components of abnormal sounds that may have obvious characteristics. In addition, the use of a single log-Mel spectrogram may not be able to fully distinguish between normal and abnormal sounds, resulting in unstable performance when used with a self-supervised method. In view of the above discussion, the present invention aims to monitor the operating status of a belt conveyor in a noisy environment by using an abnormal sound detection method based on deep learning. Summary of the invention

[0005] In view of the above discussion, the present invention proposes an abnormal sound detection method based on multi-scale feature fusion for belt conveyor belt operation detection. It takes the original sound data as input, adopts a self-supervised method for detection and recognition, and uses a convolutional neural network (CNN) to extract time features on the basis of log-Mel to make up for the abnormal information that cannot be obtained based on the log-Mel spectrum. MobileFaceNet (MFN) is used as a classifier, and the fused information is input into MFN to learn the detailed features of normal sounds. Finally, we splice the convolution information under the time and space features, and generate the final feature information, which is input into the network to determine whether a fault occurs and the type of fault.

[0006] The present invention provides an abnormal sound detection method for belt conveyor fault diagnosis, comprising the following steps:

[0007] The first step is to build a data set. The training data set and the test data set are composed of the training data and the test data in MIMII and ToyADMOS, including the fault types under different working conditions;

[0008] The second step is to use the training data set to train the preset multi-scale feature fusion abnormal sound detection network; the multi-scale feature fusion abnormal sound detection network includes: a spectrum-time feature fusion module and a time information feature extraction network module based on a convolutional neural network;

[0009] In the third step, the abnormal sound is input into the trained multi-scale feature fusion abnormal sound detection network to complete the fault diagnosis of abnormal sound detection.

[0010] Preferably, the method for training the abnormal sound detection network with multi-scale feature fusion in the second step includes: using short-time Fourier transform to obtain the number of frequency bands of the spectrum graph, and obtaining abnormal sound data according to these frequency bands; the logarithmic Mel spectrum graph converts the frequency axis of the linear spectrum graph into the Mel frequency axis through logarithmic transformation, thereby increasing resistance to the influence of environmental noise and improving the distinguishability of features;

[0011] The time information feature extraction network module is used to compensate for the missing abnormal information in the logarithmic Mel spectrum and enrich the abnormal features. Three convolution layers with one-dimensional convolution kernels are used. Different convolution kernels have different sizes. These convolution layers are used to extract the feature information of the original input sound data. The feature information includes audio thresholds, audio corresponding information in time and space. The shallow features are converted into feature maps of different scales through convolution methods of different degrees. Among them, a large kernel one-dimensional convolution is used, followed by a smaller convolution block. Each block consists of a layer normalization, a ReLU activation function and a one-dimensional convolution with a smaller kernel size. Through the combination of different convolution kernel sizes, subtle features and local information in the time series are effectively extracted.

[0012] Finally, the time axis-related convolution of the logarithmic Mel spectrum is added to the spatial information obtained by the information compensation network to get the final image and complete the training.

[0013] Preferably, after completing the second step, a feature completion operation is performed, and the expression is:

[0014] x i =σ[W i ·x(τ)+b i ]

[0015] F S =log(W M ||STFT(x)|| 2 )

[0016] Where W i represents the vector weight of the convolution kernel of size i, σ represents the activation function, b i represents the bias vector of the convolution kernel of size i, x(τ) represents the original input sound signal of monophonic channel, x i represents the output of the convolution kernel of the i-th size, i∈(1,3,5); represents the Mel filter bank, and B is the number of frequency bands of the spectrogram obtained by short-time Fourier transform (STFT).

[0017] Preferred:

[0018] x c =add(x 1 ,x 3 ,x 3 )

[0019] x z =concat(x c ,x m )

[0020] where x 1 ,x 3,x 3 represents the output of the CNN layer with the corresponding convolution kernel size, add represents the feature fusion strategy, and the feature maps are added with the same number of channels, x c represents the output of temporal feature fusion, x m represents the Log-Mel spectrum, concat represents the feature fusion strategy that does not change the size of the feature map and does not increase the number of channels; x z Represents the input features of time-spectral information fusion.

[0021] Preferably, after completing the second step, the loss in training the abnormal sound detection network with multi-scale feature fusion is measured, and the loss function is:

[0022]

[0023] This loss function maps the feature vector onto a hypersphere of radius S as the angle between the input feature and the target weight, and adds an additional angular interval m,θ between the input feature and the target weight yi +m to enhance intra-class compactness and inter-class variability.

[0024] Compared with the prior art, the present invention has the following advantages and technical effects:

[0025] In this invention, the information extraction of different branches and the elimination of abnormal noise are realized mainly through the proposed simple and effective multi-scale feature fusion module (STasd), and the abnormal detection of sound is realized by fusing features of different scales through a simple classification module (MFN). Specifically:

[0026] STasd combines time information with spatial information, extracts feature information corresponding to it in the time dimension through the logarithmic Mel spectrum graph, and extracts feature compensation missing abnormal information through the time information feature extraction network module, thus achieving mutual complementation of information and improving the feature extraction ability of the model. Through lightweight analysis through MFN, limited mobile resources are analyzed with a small amount of model parameters and computational complexity. The design of MobileFaceNet takes into account the real-time requirements and can achieve fast inference speed on mobile devices. This means that MobileFaceNet can process data quickly, perform fault diagnosis quickly and accurately, and is suitable for scenarios that require low latency. Finally, satisfactory detection results are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0028] Figure 1It is a flowchart of this application;

[0029] Figure 2 It is the overall framework diagram of this application;

[0030] Figure 3 This is a schematic diagram of the structure of the feature fusion module of this application;

[0031] Figure 4 It is a schematic diagram of the results of this application. DETAILED DESCRIPTION

[0032] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0033] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. Although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0034] like Figures 1 to 4 As shown, an abnormal sound detection method for belt conveyor fault diagnosis comprises the following steps:

[0035] The first step is to build a data set. The training data set and the test data set are composed of the training data and the test data in MIMII and ToyADMOS, including the fault types under different working conditions;

[0036] The second step is to use the training data set to train the preset multi-scale feature fusion abnormal sound detection network; the multi-scale feature fusion abnormal sound detection network includes: a spectrum-time feature fusion module and a time information feature extraction network module based on a convolutional neural network;

[0037] In the third step, the abnormal sound is input into the trained multi-scale feature fusion abnormal sound detection network to complete the fault diagnosis of abnormal sound detection.

[0038] Preferably, the method for training the abnormal sound detection network with multi-scale feature fusion in the second step includes: using short-time Fourier transform to obtain the number of frequency bands of the spectrum graph, and obtaining abnormal sound data according to these frequency bands; the logarithmic Mel spectrum graph converts the frequency axis of the linear spectrum graph into the Mel frequency axis through logarithmic transformation, thereby increasing resistance to the influence of environmental noise and improving the distinguishability of features;

[0039] The time information feature extraction network module is used to compensate for the missing abnormal information in the logarithmic Mel spectrum and enrich the abnormal features. Three convolution layers with one-dimensional convolution kernels are used. Different convolution kernels have different sizes. These convolution layers are used to extract the feature information of the original input sound data. The feature information includes audio thresholds, audio corresponding information in time and space. The shallow features are converted into feature maps of different scales through convolution methods of different degrees. Among them, a large kernel one-dimensional convolution is used, followed by a smaller convolution block. Each block consists of a layer normalization, a ReLU activation function and a one-dimensional convolution with a smaller kernel size. Through the combination of different convolution kernel sizes, subtle features and local information in the time series are effectively extracted.

[0040] Finally, the time axis-related convolution of the logarithmic Mel spectrum is added to the spatial information obtained by the information compensation network to get the final image and complete the training.

[0041] Preferably, after completing the second step, a feature completion operation is performed, and the expression is:

[0042] x i =σ[W i ·x(τ)+b i ]

[0043] F S =log(W M ||STFT(x)|| 2 )

[0044] Where W i represents the vector weight of the convolution kernel of size i, σ represents the activation function, b i represents the bias vector of the convolution kernel of size i, x(τ) represents the original input sound signal of monophonic channel, x i represents the output of the convolution kernel of the i-th size, i∈(1,3,5); represents the Mel filter bank, and B is the number of frequency bands of the spectrogram obtained by short-time Fourier transform (STFT).

[0045] Preferred:

[0046] x c =add(x 1 ,x 3 ,x 3 )

[0047] x z =concat(x c ,x m )

[0048] where x 1 ,x 3,x 3 represents the output of the CNN layer with the corresponding convolution kernel size, add represents the feature fusion strategy, and the feature maps are added with the same number of channels, x c represents the output of temporal feature fusion, x m represents the Log-Mel spectrum, concat represents the feature fusion strategy that does not change the size of the feature map and does not increase the number of channels; x z Represents the input features of time-spectral information fusion.

[0049] Preferably, after completing the second step, the loss in training the abnormal sound detection network with multi-scale feature fusion is measured, and the loss function is:

[0050]

[0051] This loss function maps the feature vector onto a hypersphere of radius S as the angle between the input feature and the target weight, and adds an additional angular interval m,θ between the input feature and the target weight yi +m to enhance intra-class compactness and inter-class variability.

[0052] Examples, such as Figures 1 to 4 As shown, the abnormal sound detection method should focus on feature extraction, feature recognition, and sound denoising. In view of the fact that the multi-scale network structure can well integrate low-level features and high-level features, abnormal sound detection is achieved by processing features of different scales and then progressively integrating and reprocessing them. The present invention proposes an abnormal sound detection network composed of a multi-scale feature extraction module and a feature clustering analysis module as main enhancement modules.

[0053] In order to make full use of the abnormal information in the sound, this example extracts the features of the input sound information in the spatial dimension. By converting the audio signal into a spectrogram, and then filtering and logarithmically transforming the frequency using the Mel scale, the features are more consistent with the auditory characteristics of the human ear. This conversion provides a two-dimensional representation of time and frequency, which helps capture the characteristics of transient events and abnormal sounds. In addition, a convolutional neural network is used to extract features from the information lost in the logarithmic Mel spectrogram. IN ∈R H×W×3 Extract feature X through three layers of convolution 特征 ∈R H×W×C ,While reducing the feature information, effective information is obtained and abnormal noise is removed.

[0054] like Figure 3As shown in the figure, MobileFaceNet uses GDConv to replace the traditional global average pooling (GAP) layer. The convolution kernel size of GDConv is equal to the size of the input feature map, and each channel performs convolution operations independently. This allows GDConv to learn the importance weights of features at different positions, thereby more effectively extracting the key features of the sound. MobileFaceNet is based on the network structure of MobileNetV2 and has been optimized. It uses smaller bottlenecks and adopts the PReLu activation function to improve the efficiency and performance of the model. The basic structure of MobileFaceNet is MobileNetV2, which uses an efficient depthwise separable convolution (Depthwise SeparableConvolution) structure to decompose the standard convolution into depthwise convolution and point-by-point convolution, thereby reducing the amount of calculation and the number of parameters. The bottlenecks in MobileFaceNet are smaller than those in MobileNetV2, which further reduces the amount of calculation and the number of parameters.

[0055] The loss function plays a role in guiding model training, measuring reconstruction error, improving feature extraction, and processing specific task requirements in abnormal sound detection. In abnormal sound detection, commonly used loss functions include structural loss, reconstruction loss, etc.

[0056] Structural loss (ArcFaceLoss): ArcFace introduces angle boundaries, which are measured by the cosine of the angle, and pays more attention to intra-class compactness and inter-class separability. It can help the model learn more discriminative features and is particularly effective for complex classification tasks:

[0057]

[0058] Where L1: represents the loss function of a single sample. m: represents the total number of training samples. n: represents the number of categories. wi: represents the weight vector of the i-th category. bi,: represents the bias of the i-th category. xi: represents the feature vector of the i-th sample. T: represents the transposition operation. e: represents the exponential function. ∑: represents the summation operation. log: represents the logarithmic function. For each sample, the probability of it belonging to each category is calculated and compared with the probability of the true label to obtain a loss value. Finally, the loss values ​​of all samples are averaged to obtain the final cross entropy loss.

[0059] Reconstruction loss is often used in data dimensionality reduction and feature extraction. Its formula is as follows:

[0060]

[0061] Where L: represents the loss or error of the model. It measures the performance of the model, and lower values ​​indicate better performance of the model. N: represents the total number of samples or examples used to train the model. Σ: This is the summation symbol, which means that we are adding up a series of values. i=1: This indicates the starting point of the summation. We will add from the first sample (i=1) to the Nth sample. x: represents the input to the model for each sample. This is the data that the model is trying to learn from. f(x): represents the first transformation or function applied to the input x. This can be any type of processing, such as feature extraction or simple mathematical operations. g(f(x)): represents the second transformation or function applied to the result of the first transformation. This can be another processing layer or a prediction function. xg(f(x)): This calculates the difference between the original input and the output of the model after the two transformations. This difference represents the error or bias made by the model.

[0062] The proposed simple and effective multi-scale feature fusion module (STasd) is used to extract information from different branches and eliminate abnormal noise. At the same time, a simple classification module (MFN) is used to fuse features of different scales to achieve sound anomaly detection.

[0063] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An abnormal sound detection method for belt conveyor fault diagnosis, characterized in that: The steps include: The first step is to build a data set. The training data set and the test data set are composed of the training data and the test data in MIMII and ToyADMOS, including the fault types under different working conditions; The second step is to use the training data set to train the preset multi-scale feature fusion abnormal sound detection network; The abnormal sound detection network with multi-scale feature fusion includes: a spectrum-time feature fusion module and a time information feature extraction network module based on a convolutional neural network; In the third step, the abnormal sound is input into the trained multi-scale feature fusion abnormal sound detection network to complete the fault diagnosis of abnormal sound detection.

2. The abnormal sound detection method for belt conveyor fault diagnosis according to claim 1, characterized in that: The method for training the abnormal sound detection network with multi-scale feature fusion in the second step includes: Short-time Fourier transform is used to obtain the number of frequency bands of the spectrum graph, and abnormal sound data is obtained based on these frequency bands; the logarithmic Mel spectrum graph converts the frequency axis of the linear spectrum graph into the Mel frequency axis through logarithmic transformation, thereby increasing resistance to the influence of environmental noise and improving the distinguishability of features; The time information feature extraction network module is used to compensate for the missing abnormal information in the logarithmic Mel spectrum and enrich the abnormal features. Three convolution layers with one-dimensional convolution kernels are used. Different convolution kernels have different sizes. These convolution layers are used to extract the feature information of the original input sound data. The feature information includes audio thresholds, audio corresponding information in time and space. The shallow features are converted into feature maps of different scales through convolution methods of different degrees. Among them, a large kernel one-dimensional convolution is used, followed by a smaller convolution block. Each block consists of a layer normalization, a ReLU activation function and a one-dimensional convolution with a smaller kernel size. Through the combination of different convolution kernel sizes, subtle features and local information in the time series are effectively extracted. Finally, the time axis-related convolution of the logarithmic Mel spectrum is added to the spatial information obtained by the information compensation network to get the final image and complete the training.

3. The abnormal sound detection method for belt conveyor fault diagnosis according to claim 1, characterized in that: After completing the second step, perform feature completion operation, the expression is: x i =σ[W i ·x(τ)+b i ] F S =log(W M ·||STFT(x)|| 2 ) Where W i represents the vector weight of the convolution kernel of size i, σ represents the activation function, b i represents the bias vector of the convolution kernel of size i, x(τ) represents the original input sound signal of monophonic channel, x i represents the output of the convolution kernel of the i-th size, i∈(1,3,5); represents the Mel filter bank, and B is the number of frequency bands of the spectrogram obtained by short-time Fourier transform (STFT).

4. The abnormal sound detection method for belt conveyor fault diagnosis according to claim 2, characterized in that: x c =add(x1,x3,x3) x z =concat(x c ,x m ) Among them, x1, x3, x3 represent the output of the CNN layer with the corresponding convolution kernel size, add represents the feature fusion strategy, and the feature maps are added with the same number of channels. c represents the output of temporal feature fusion, x m represents the Log-Mel spectrum, concat represents the feature fusion strategy that does not change the size of the feature map and does not increase the number of channels; x z Represents the input features of time-spectral information fusion.

5. The abnormal sound detection method for belt conveyor fault diagnosis according to claim 1, characterized in that: After completing the second step, the loss in training the abnormal sound detection network with multi-scale feature fusion is measured, and the loss function is: This loss function maps the feature vector onto a hypersphere of radius S as the angle between the input feature and the target weight, and adds an additional angular interval m,θ between the input feature and the target weight yi +m to enhance intra-class compactness and inter-class variability.

Images