Carrier roller fault monitoring method based on sound multi-feature fusion

Abstract

The invention relates to the technical field of industrial equipment state monitoring and fault diagnosis, in particular to a carrier roller fault monitoring method based on sound multi-feature fusion for a carrier roller of a belt conveyor. The method comprises the following steps: firstly, collecting a sound signal when the carrier roller runs, and segmenting the sound signal into segments with fixed duration; then, for each sound segment, a logarithmic Mel spectrum, a Mel frequency cepstral coefficient graph and a spectral contrast graph are extracted in parallel. The feature images are superposed and fused into a three-channel feature image after being subjected to independent channel normalization processing and size unification. And finally, inputting the fused three-channel feature image into a convolutional neural network for training and reasoning of a fault classification model, and realizing identification of various carrier roller fault types. According to the method, the defects of an existing carrier roller fault monitoring method in the aspects of recognition accuracy, noise immunity, fault type subdivision capability, intelligent degree and the like are overcome, and the accuracy and robustness of fault monitoring can be remarkably improved by fusing multiple complementary acoustic features and utilizing the powerful automatic learning capability of the deep learning model.

Description

Technical Field

[0001] This invention relates to the field of industrial equipment condition monitoring and fault diagnosis technology, and in particular to a method for monitoring idler roller faults by sound multi-feature fusion for belt conveyor idlers. Background Technology

[0002] Belt conveyors are indispensable continuous material transport equipment in industries such as coal mines, ports, power plants, and metallurgy. Idler rollers, as the core load-bearing and guiding components of belt conveyors, are numerous, operate in harsh environments, and are typical wear parts. Therefore, timely and effective fault monitoring and diagnosis of idler rollers are of great significance for ensuring the safe and stable operation of belt conveyors and reducing maintenance costs.

[0003] Currently, the main methods for detecting idler roller faults can be categorized as follows: Manual inspection: This method relies mainly on the experience of the inspectors, who judge the condition of the idlers by listening and observing. This method is highly subjective, inefficient, labor-intensive, difficult to monitor in real time, and prone to missed inspections and misjudgments.

[0004] Monitoring based on single signal features: Researchers have attempted to use signals such as vibration, temperature, and acoustics for monitoring. For example, vibration signal analysis methods identify faults by detecting changes in the frequency and amplitude of idler roller vibration; temperature monitoring methods based on infrared thermal imaging identify faults by detecting abnormal temperature rises in idler rollers. In acoustic monitoring, early methods relied heavily on simple time-domain features (such as root mean square value and kurtosis) or frequency-domain features (such as spectral peaks and power spectra). However, these methods based on single physical quantities or simple features often suffer from low accuracy and robustness in complex industrial noise environments, are insensitive to early and weak faults, and struggle to distinguish between various subdivided fault types. For instance, some studies have indicated that single-frame Mel-frequency cepstral coefficients (MFCC) features are easily affected by noise, and research from Shanghai Jiao Tong University has proposed a scheme combining multiple analysis methods to improve diagnostic effectiveness.

[0005] Traditional machine learning methods: Some studies employ traditional machine learning algorithms such as Support Vector Machines (SVM) and Artificial Neural Networks (ANN) for fault classification. These methods offer improvements over simple thresholding methods, but their performance is highly dependent on manually designed features. The quality of feature extraction directly affects the final diagnostic results, and the model's generalization ability is limited.

[0006] Challenges of existing sound monitoring: The acoustic environment in industrial sites (such as underground coal mines) is extremely complex, with various equipment operating noises, material impact noises, and signal propagation path attenuation. These factors make it very difficult to effectively extract weak idler roller fault characteristic signals from background noise. A single acoustic feature is often insufficient to comprehensively and accurately characterize the true operating state of the idler roller.

[0007] Given the above background, there is an urgent need for an algorithm that can effectively integrate multiple complementary acoustic features and utilize the powerful automatic feature learning and classification capabilities of deep learning models to achieve intelligent and high-precision monitoring of various fault types of idlers. Summary of the Invention

[0008] This invention aims to overcome the following major problems existing in current idler roller fault monitoring technologies: Insufficient accuracy and robustness: Single features or simple feature combinations are difficult to accurately identify idler roller faults, especially early and minor faults, in industrial environments with strong noise and multiple interferences.

[0009] Limited ability to subdivide fault types: Most existing methods can only determine whether the idler roller has failed, or can only identify a few typical faults. It is difficult to distinguish between a variety of specific fault types in detail, which is not conducive to accurate maintenance.

[0010] Feature extraction relies on human experience: Traditional machine learning methods rely on manual design and selection of features, which is time-consuming and labor-intensive, and it is difficult to find the optimal combination of features, which limits the performance and generalization ability of the model.

[0011] Low level of intelligence: Manual inspection and some semi-automatic monitoring systems still require a lot of manual intervention, which cannot meet the needs of efficient, real-time and automated monitoring of large-scale, long-distance belt conveyor systems.

[0012] Specifically, the core technical problem to be solved by this invention is: how to effectively extract and integrate multiple complementary acoustic features that can comprehensively characterize the operating state of idlers from a complex industrial acoustic environment, and how to use a deep learning model to automatically learn the complex mapping relationship between these features and various types of common idler faults, so as to achieve high-precision and high-robust fault classification.

[0013] The present invention provides a method for monitoring idler roller faults by fusing multiple sound features, comprising the following steps: 1) Collect the sound signal during the operation of the idler roller, and divide the sound signal into segments of fixed duration to obtain multiple sound segments; 2) Extract three acoustic feature maps for each sound segment: log-Mel spectrogram, Mel frequency cepstral coefficient map, and spectral contrast map; 3) Normalize the three acoustic feature maps of each sound segment and adjust them to two-dimensional images of the same resolution to obtain normalized logarithmic Mel spectrogram, normalized Mel frequency cepstral coefficient map, and normalized spectral contrast map. 4) The normalized logarithmic Mel spectrum, the normalized Mel frequency cepstral coefficients, and the normalized spectral contrast diagram are used as the R, G, and B channels of the three-channel image, respectively, and fused to construct an acoustic feature color image. 5) Input the acoustic feature color image into the convolutional neural network model, identify the acoustic feature color image through the convolutional neural network model, and output the roller fault classification result corresponding to the sound segment.

[0014] Preferably, in step 3), the normalization process employs a minimum-maximum normalization method, and its normalization process can be described as follows: In the above formula, X norm It is the result of normalization; X is the pixel value of the original feature map; min(X) is the minimum value of the pixels in the original feature map; max(X) is the maximum value of the pixels in the original feature map.

[0015] Preferably, in step 2), the extraction process of the log-Mel spectrum includes the following steps: 21) The short-time Fourier transform of the original sound signal can be described by the following formula: ; In the above formula, x(t) is the original continuous-time signal, and w(t−m) is a window function centered at m. ω is the frequency variable, and m is the position of the time sliding window; 22) Calculate the power spectrum for each frame, which can be described by the formula: ; In the above formula, k is the frequency index, m is the time frame index, X(k,m) represents the complex amplitude of the k-th frequency point of the m-th frame obtained by the short-time Fourier transform, and P(k,m) represents the power spectrum. 23) By weighting and summing the power spectrum using the Mel filter bank, the Mel spectrum is obtained as follows: ; In the above formula, S(m,t) represents the m-th Mel-band energy of the t-th frame, m is the Mel filter index, k is the frequency index, t is the time index, N is the number of short-time Fourier transform frequency points, and H... m (k) is the weighting coefficient of the m-th Mel filter at frequency point k; 24) Taking the logarithm of the Mel spectrum yields the logarithmic Mel spectrum: ; In the above formula, LogMel(m,t) is the logarithmic energy of the m-th Mel band in the t-th frame, and ϵ is a very small constant to prevent log(0); The extraction process of the Mel frequency cepstral coefficient map includes: 25) Perform Discrete Cosine Transform (DCT) on each frame based on the log-Mel spectrum to obtain the cepstral coefficients. The calculation formula is as follows: ; In the above formula, M is the number of Mel filters, m is the number of extracted coefficients, and MFCC(n,t) is the nth Mel frequency cepstral coefficient of the tth frame. The contrast in the spectral contrast diagram can be described as: In the above formula, B b Let C represent the set of frequency indices for the b-th frequency band, where ϵ is a very small constant to prevent division by zero. b (t) is the spectral contrast of the b-th frequency band at time frame t.

[0016] Preferably, in step 5), the convolutional neural network is a ResNet18 convolutional neural network.

[0017] The ResNet18 convolutional neural network is constructed based on: an initial convolutional layer and a max-pooling layer; four stages of residual blocks, each stage containing two residual blocks, each residual block consisting of two convolutional layers; each convolutional layer followed by a normalization layer and a ReLU activation function; a global average pooling layer; and a fully connected layer as a classifier, followed by a Softmax activation function to output the probability distribution of each category. The training parameters of the neural network include: 3 input channels, a batch size of 24, an initial learning rate of 0.1, a minimum learning rate of 1e-6, 300 training epochs, cosine annealing as the learning rate scheduling strategy, and 5 learning rate warm-up epochs. The output includes 11 fault categories: abnormal motor noise, damaged roller rubber, worn bearing seals, overly tight cleaner, material adhering to the idler roller skin, abnormal noise from worn idler roller shell, abnormal idler roller noise, worn idler roller bearing steel balls, damaged idler roller bearing, dust ingress into the bearing, and normal operation. The ReLU activation function is defined as follows: ; In the above formula, x represents the input value, and ReLU represents the output value; The Softmax activation function is defined as follows: ; In the above formula, Softmax(z) i) is the output value of the Softmax function for the i-th input, representing the probability of that class, z. i It is the i-th component of vector z, z j Let be the j-th component of z, where j is the summation index variable and K is the number of categories.

[0018] Compared with existing technologies, the sound multi-feature fusion algorithm for idler roller fault detection proposed in this invention has the following significant advantages: Significantly improves fault identification accuracy and robustness: By fusing three acoustic features—log-Mel spectrum (reflecting energy and pitch), MFCC plot (reflecting timbre), and spectral contrast plot (reflecting spectral clarity and noise)—it can more comprehensively capture subtle acoustic changes caused by idler roller faults from multiple dimensions. Combined with the powerful nonlinear feature learning and classification capabilities of the ResNet18 deep convolutional neural network, it can effectively distinguish different fault modes, significantly improving the accuracy and robustness of fault identification in complex industrial noise environments. Multi-feature fusion helps overcome the limitations of single features under specific noise or operating conditions, outperforming single-frame MFCC.

[0019] Achieving fine-grained classification of multiple fault types: This algorithm can effectively distinguish up to 11 specific idler roller-related fault types and normal operating states. This fine-grained fault diagnosis capability provides a more accurate basis for subsequent equipment maintenance decisions (such as judging the severity of the fault, identifying the parts to be repaired, and arranging maintenance plans), which helps to achieve predictive maintenance and precision repair.

[0020] Achieving intelligent and automated monitoring: The entire monitoring process, from sound signal acquisition to final fault type output, achieves end-to-end automated processing. The application of deep learning models avoids tedious manual feature engineering, reduces human intervention, and improves monitoring efficiency and intelligence, making it particularly suitable for large-scale, long-distance, unattended belt conveyor systems.

[0021] Effectiveness of Feature Extraction and Representation: Mapping acoustic features and processing them using the ResNet18 network, which has achieved great success in image recognition, represents an innovative cross-modal application. This method fully leverages mature technologies and model architectures in computer vision, transforming abstract acoustic features into intuitive image patterns, enabling deep learning models to learn and recognize fault patterns more effectively. Enhanced Sensitivity to Early and Subtle Faults: The fusion of multiple complementary features, particularly the introduction of features sensitive to signal details such as spectral contrast, helps capture subtle acoustic signal changes generated by the idler roller in the early stages of a fault, potentially enabling early warning of faults and preventing more severe losses due to fault escalation. Attached Figure Description

[0022] Figure 1This is a flowchart of one embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram illustrating the process of sound signal feature extraction and three-channel fusion.

[0024] Figure 3 This is a schematic diagram of the ResNet18 network structure. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0026] The present invention discloses a method for monitoring idler roller faults by fusing multiple sound features, comprising the following steps: 1) Collect the sound signal during the operation of the idler roller, and divide the sound signal into segments of fixed duration to obtain multiple sound segments; 2) Extract three acoustic feature maps for each sound segment: log-Mel spectrogram, Mel frequency cepstral coefficient map, and spectral contrast map; The extraction process of the logarithmic Mel spectrum includes the following steps: 21) The short-time Fourier transform of the original sound signal can be described by the following formula: ; In the above formula, x(t) is the original continuous-time signal, w(t−m) is a window function centered at m, ω is the frequency variable, and m is the position of the time sliding window; 22) Calculate the power spectrum for each frame, which can be described by the formula: ; In the above formula, k is the frequency index, m is the time frame index, X(k,m) represents the complex amplitude of the k-th frequency point of the m-th frame obtained by the short-time Fourier transform, and P(k,m) represents the power spectrum. 23) By weighting and summing the power spectrum using the Mel filter bank, the Mel spectrum is obtained as follows: ; In the above formula, S(m,t) represents the m-th Mel-band energy of the t-th frame, m is the Mel filter index, k is the frequency index, t is the time index, N is the number of short-time Fourier transform frequency points, and H... m (k) is the weighting coefficient of the m-th Mel filter at frequency point k; 24) Taking the logarithm of the Mel spectrum yields the logarithmic Mel spectrum: ; In the above formula, LogMel(m,t) is the logarithmic energy of the m-th Mel band in the t-th frame, and ϵ is a very small constant to prevent log(0); The extraction process of the Mel frequency cepstral coefficient map includes: 25) Perform Discrete Cosine Transform (DCT) on each frame based on the log-Mel spectrum to obtain the cepstral coefficients. The calculation formula is as follows: ; In the above formula, M is the number of Mel filters, m is the number of extracted coefficients, and MFCC(n,t) is the nth Mel frequency cepstral coefficient of the tth frame. The contrast in the spectral contrast diagram can be described as: ; In the above formula, B b Let C represent the set of frequency indices for the b-th frequency band, where ϵ is a very small constant to prevent division by zero. b (t) is the spectral contrast of the b-th frequency band at time frame t.

[0027] 3) Normalize the three acoustic feature maps of each sound segment and adjust them to two-dimensional images of the same resolution to obtain normalized logarithmic Mel spectrogram, normalized Mel frequency cepstral coefficient map, and normalized spectral contrast map. The normalization process employs a minimum-maximum normalization method, and its normalization process can be described as follows: ; In the above formula, X norm It is the result of normalization; X is the pixel value of the original feature map; min(X) is the minimum value of the pixels in the original feature map; max(X) is the maximum value of the pixels in the original feature map.

[0028] 4) The normalized logarithmic Mel spectrum, the normalized Mel frequency cepstral coefficients, and the normalized spectral contrast diagram are used as the R, G, and B channels of the three-channel image, respectively, and fused to construct an acoustic feature color image. 5) Input the acoustic feature color image into the convolutional neural network model, identify the acoustic feature color image through the convolutional neural network model, and output the roller fault classification result corresponding to the sound segment.

[0029] The convolutional neural network is a ResNet18 convolutional neural network.

[0030] The ResNet18 convolutional neural network is constructed based on: an initial convolutional layer and a max-pooling layer; four stages of residual blocks, each stage containing two residual blocks, each residual block consisting of two convolutional layers; each convolutional layer followed by a normalization layer and a ReLU activation function; a global average pooling layer; and a fully connected layer as a classifier, followed by a Softmax activation function to output the probability distribution of each category. The training parameters of the neural network include: 3 input channels, a batch size of 24, an initial learning rate of 0.1, a minimum learning rate of 1e-6, 300 training epochs, cosine annealing as the learning rate scheduling strategy, and 5 learning rate warm-up epochs. The output includes 11 fault categories: abnormal motor noise, damaged roller rubber, worn bearing seals, overly tight cleaner, material adhering to the idler roller skin, abnormal noise from worn idler roller shell, abnormal idler roller noise, worn idler roller bearing steel balls, damaged idler roller bearing, dust ingress into the bearing, and normal operation. The ReLU activation function is defined as follows: ; In the above formula, x represents the input value, and ReLU represents the output value; The Softmax activation function is defined as follows: ; In the above formula, Softmax(z) i ) is the output value of the Softmax function for the i-th input, representing the probability of that class, z. i It is the i-th component of vector z, z j Let be the j-th component of z, where j is the summation index variable and K is the number of categories.

[0031] Specifically, the sound multi-feature fusion algorithm for idler roller fault detection described in this invention mainly includes the following steps: (1) Sound signal acquisition and segmentation: The sound signal of the idler rollers during operation is acquired in real time by directional microphones deployed near the idler rollers of the belt conveyor. The continuous sound signal is segmented into segments of fixed duration, for example, each segment is 3 seconds long, as the basic unit for subsequent analysis.

[0032] (2) Multi-dimensional acoustic feature extraction: For each 3-second sound segment, the following three complementary acoustic feature maps are extracted in parallel: Log Mel Spectrogram: This feature reflects how the energy distribution of a sound signal changes over time on the Mel frequency scale, simulating the auditory perception characteristics of the human ear and being sensitive to changes in pitch and energy of sound.

[0033] Mel-frequency cepstral coefficients (MFCC plot): MFCCs are coefficients obtained by performing a discrete cosine transform (DCT) on the logarithmic Mel spectrum. They effectively characterize the timbre of a sound and are commonly used in speech recognition and voiceprint recognition. Arranging them frame by frame forms an MFCC plot.

[0034] Spectral Contrast: This feature reflects the clarity, harmonic structure, and noise level of the spectrum by calculating the contrast between peak and valley energies within different frequency bands. High contrast typically corresponds to a clear narrowband signal, while low contrast may indicate broadband noise or a blurred signal.

[0035] (3) Feature fusion and three-channel image construction: Channel normalization: The extracted three types of two-dimensional feature maps (log-Mel spectrum, MFCC map, and spectral contrast map) are subjected to in-channel numerical normalization (e.g., scaling their numerical range to [0, 1] or [-1, 1]) to eliminate the influence of differences in numerical scale between different feature maps.

[0036] Size unification (Resize): The three normalized feature maps are uniformly adjusted to the same preset size, such as 640x640 pixels, so that they can be fused into a standard image input later.

[0037] Three-channel image overlay: Three resized feature maps are overlaid and fused into a three-channel color image in a predetermined order. For example, the log-Mel spectrum is used as the R (red) channel, the MFCC image as the G (green) channel, and the spectral contrast image as the B (blue) channel. In this way, each pixel of each fused image contains information from three different acoustic dimensions.

[0038] (4) Training and inference of fault classification model based on ResNet18 network: Classification model selection: The ResNet18 (residual network - 18 layers) convolutional neural network was adopted as the core fault classification model. ResNet18 was chosen because of its effective residual learning mechanism, relatively few parameters, and excellent performance on image classification tasks.

[0039] Model input: The input to the model is the 640x640x3 three-channel fused feature image constructed in the previous step.

[0040] Model Output: The model's output layer is designed to classify the probabilities of various preset idler roller fault types (including normal states). In this invention, 11 categories are defined, specifically including: abnormal motor noise, damaged roller coating, worn bearing seals, overly tight cleaner, material adhering to the idler roller skin, abnormal noise from worn idler roller shell, abnormal idler noise (specifically referring to situations where the cause is unclear but the sound is abnormal), worn idler roller bearing balls, damaged idler roller bearings (such as jamming or breakage), dust ingress into the bearing, and normal operation.

[0041] Model Training: Supervised learning training is performed using a dataset of audio samples containing the 11 known labels mentioned above. During training, specific training parameters are used, such as: number of input channels (`--in-chans 3`), batch size (`--batch-size 24`), initial learning rate (`--lr 0.1`), number of warmup epochs (`--warmup-epochs 5`), total number of training epochs (`--epochs 300`), minimum learning rate (`--min-lr 1e-6`), learning rate scheduling strategy (`--schedcosine`, cosine annealing), and base batch size for learning rate adjustment (`--lr-base-size 24`).

[0042] Fault Reasoning / Monitoring: For real-time acquired idler roller sound segments, after undergoing the same feature extraction, fusion, and three-channel image construction process, they are input into the already trained ResNet18 model. The model will output the probability distribution of the sound segment corresponding to 11 state categories, and take the category with the highest probability as the fault state diagnosis result of the current idler roller.

[0043] Example 1 The idler roller fault monitoring algorithm of this invention can be widely applied in industrial scenarios using belt conveyors in the iron and steel metallurgy industry. It can be used to monitor the idler rollers of belt conveyors used for material conveying in processes such as sintering, ironmaking, and steelmaking, thereby ensuring the continuity of the production process.

[0044] In one specific embodiment, an inspection robot equipped with a directional microphone can be installed above a main coal conveyor belt. The robot's acquisition unit uploads a 3-second audio clip to a server via a wireless network. After processing into a three-channel fused feature image, the server runs a pre-trained ResNet18 model for real-time inference. When a specific fault is detected on a roller at a certain location and the probability exceeds the alarm threshold, the system automatically highlights the fault location and type on the monitoring interface and can send alarm information to maintenance personnel to guide them in performing quick and accurate repairs.

[0045] To verify the effectiveness of this invention, a roller sound dataset containing the aforementioned 11 types of faults can be constructed. Comparative experiments will be conducted with methods using only single features (such as MFCC or LogMel spectrograms only) or with traditional machine learning methods (such as SVM + artificial features). Evaluation metrics can include the recognition accuracy, overall accuracy, precision, recall, and F1 score for each type of fault. It is expected that the multi-feature fusion algorithm combined with ResNet18 proposed in this invention will demonstrate superiority across all evaluation metrics, particularly in robustness under complex noise environments and the ability to identify subdivided fault types.

[0046] I. Algorithm Flow Overview like Figure 1 As shown, the core process of the sound multi-feature fusion algorithm for idler roller fault detection described in this invention can be summarized as follows: (1) Sound signal acquisition and segmentation: Use a directional microphone to acquire the sound signal of the belt conveyor roller during operation, and divide the continuous audio stream into independent audio segments with a fixed duration of 3 seconds.

[0047] (2) Parallel acoustic feature extraction: For each audio segment, three acoustic feature maps are extracted simultaneously: Log Mel Spectrogram, Mel frequency cepstral coefficient map (MFCC map), and Spectral Contrast map.

[0048] (3) Feature map preprocessing: Normalize the values ​​of the three feature maps in each channel. Then, adjust the size of the three normalized feature maps to the preset size of 640x640 pixels.

[0049] (4) Three-channel feature image fusion: The three preprocessed feature maps are superimposed and fused into a three-channel image. The R channel is given a log-Mel spectrum, the G channel is given an MFCC map, and the B channel is given a spectral contrast map.

[0050] (5) ResNet18 model training and inference: Training phase: The ResNet18 classification model was trained using a fusion feature image dataset with labels (11 types of faults and normal states).

[0051] Inference phase: The real-time generated fused feature image is input into the trained ResNet18 model, and the fault classification result corresponding to the audio segment is output.

[0052] (6) Fault result output and decision: Based on the classification probability output by the model, determine the current operating status or fault type of the idler roller, and trigger alarms or guide maintenance accordingly.

[0053] The implementation details of each major module will be explained in detail below.

[0054] II. Sound Data Acquisition and Preprocessing Module Sound acquisition equipment: Microphones (arrays) or directional pickups can be used. The sound acquisition equipment should be deployed near or towards the belt conveyor rollers to capture the sound signals emitted by the rollers as clearly as possible during operation. The audio signal sampling rate is set to 16000 Hz.

[0055] Signal segmentation: The continuously acquired audio stream is divided into independent audio segments of fixed duration of 3 seconds. Each segment will be used as a sample for subsequent feature extraction and analysis.

[0056] Signal preprocessing: Filtering: A bandpass filter can be designed based on the typical frequency range of the idler roller fault sound signal to filter out extreme high-frequency noise components (>1600HZ) or low-frequency noise components (<100HZ) that are unrelated to the fault characteristics.

[0057] Noise Reduction: For industrial environments with strong noise, some preliminary noise reduction algorithms, such as spectral subtraction, Wiener filtering, or wavelet denoising techniques, can be used to improve the signal-to-noise ratio. However, it should be noted that excessive noise reduction may result in the loss of useful fault information. This invention mainly relies on the noise resistance of subsequent multi-feature fusion and deep learning models.

[0058] III. Acoustic Feature Extraction Module (1) Log-Mel spectrum is a time-frequency diagram that simulates human auditory perception. It mainly includes four steps: short-time Fourier transform (STFT), power spectrum calculation, Mel filter bank mapping and log-transform.

[0059] Let the input audio signal be x(n). The short-time Fourier transform of this signal after window function weighting is: Where: X(k,m) is the k-th frequency point of the m-th frame; w(n) is the window function (Hamming window); H is the frame shift; N is the frame length; k=0,1,...,N−1.

[0060] Then calculate the power spectrum for each frame: The power spectrum is then weighted and summed using a set of m Mel filters Hm(k): Finally, logarithmic compression is performed to obtain the log-Mel spectrum: Where 𝜖 is a minimal constant to prevent logarithmic overflow.

[0061] (2) MFCC is based on the log-Mel spectrogram and performs Discrete Cosine Transform (DCT) to remove the correlation between spectra, thereby extracting low-dimensional auditory features. The calculation method is to perform Discrete Cosine Transform on each frame t after obtaining the log-Mel spectrogram LogMel(m,t): Where M is the number of Mel filters; m is the number of extracted coefficients (13); MFCC(n,t) is the nth MFCC coefficient in the tth frame.

[0062] (2) Spectral contrast reflects the degree of contrast between the maximum and minimum amplitudes in each frequency band, and is often used to distinguish between tuned sound and untuned noise.

[0063] Let the STFT amplitude spectrum of a frame be |X(k,t)|, and divide the frequency into B sub-bands. For the b-th frequency band, its spectral contrast is defined as: Among them B b Represents the set of frequency indices for the b-th frequency band; ϵ is a very small constant to prevent division by zero; C b (t) is the spectral contrast of the b-th frequency band at time frame t. The entire spectral contrast diagram is composed of all C... b The matrix is ​​composed of (t) and has a dimension of B*T.

[0064] IV. Multi-feature fusion module like Figure 2 As shown, after extracting the three types of two-dimensional acoustic feature maps, the following fusion process is performed: (1) First, perform channel normalization. For each of the three feature maps mentioned above, independently normalize its value to the range [0, 1]. This can be achieved through min-max normalization: ; Here, X represents any feature map. If a feature map contains all zeros or has an extremely small value, a minimum value needs to be added to the denominator. Normalization can eliminate the differences in numerical ranges between different feature maps, making them have similar contribution weights when subsequently input into the neural network.

[0065] (2) Then, size unification is performed. Since the original dimensions of the three feature maps are different, in order to adapt to the standard input of the image classification network, they need to be uniformly adjusted to the same preset size (640x640 pixels). This is achieved by using the image processing library (OpenCV's cv2.resize). The interpolation method is selected as bilinear interpolation or bicubic interpolation to ensure image quality.

[0066] (3) Finally, construct a three-channel image and assign the three feature maps after resizing in the previous step to the three channels of this three-channel image respectively: we define the R channel as S_log_mel_resized*255, the G channel as mfccs_resized*255, and the B channel as spectral_contrast_resized*255.

[0067] Finally, this image represents the final feature representation input into the ResNet18 model.

[0068] V. Fault Classification Model Module Reasons for model selection: ResNet18 is a classic convolutional neural network architecture. The reasons for choosing it include: (1) It uses a residual learning mechanism, which can solve the gradient vanishing / exploding problem in deep network training, making it possible to train deeper networks and learn more complex features.

[0069] (2) Considering the balance between performance and efficiency, compared with deeper networks (ResNet50 / 101), ResNet18 has fewer parameters and lower computational complexity, making it suitable for small-scale data scenarios and less prone to overfitting; it trains faster and consumes fewer resources during inference, while still achieving excellent performance on many image classification tasks, making it suitable for deployment in industrial scenarios where computational resources may be limited.

[0070] A brief description of network structure: such as Figure 3 As shown, ResNet18 mainly consists of the following parts: An initial convolutional layer and a max pooling layer; four stages of residual blocks, each stage containing two residual blocks, each residual block consisting of two convolutional layers; each convolutional layer is typically followed by a batch normalization layer and a ReLU activation function; a global average pooling layer; and a fully connected layer as a classifier.

[0071] Input layer: Receives a 640x640x3 three-channel fused feature image obtained after processing by the above multi-feature fusion module.

[0072] Output layer: The number of output neurons in the fully connected layer is set to 11, corresponding to the 11 types of idler roller faults (including normal state) defined in this invention. This fully connected layer is followed by a Softmax activation function to convert the output into a probability distribution for each type.

[0073] ; VI. Model Training and Inference Module Dataset Construction: Fault Category Definition: This invention aims to identify the following 11 types of idler roller operating states / fault types: abnormal motor noise, damaged roller coating, worn bearing seals, overly tight cleaner, material adhering to the idler roller skin, abnormal noise due to wear of the idler roller shell, abnormal noise of the idler roller, worn steel balls in the idler roller bearing, damaged idler roller bearing, dust entering the bearing, and normal operation.

[0074] Data Source: Constructing such a diverse dataset of fault sounds is crucial to the success of this algorithm. The data comes from laboratory-simulated faults and field-collected known fault data.

[0075] Laboratory simulation of faults involves simulating various idler roller fault states on a test bench and collecting sound data. For example, a 30m belt conveyor test platform was built in the research of Shanghai University to simulate faults such as idler roller jamming, bearing failure, and breakage.

[0076] On-site collection of known fault data refers to collecting the operating sound of a specific idler roller when a particular fault is confirmed in an actual industrial setting. This requires close coordination with on-site maintenance records.

[0077] Data partitioning: The three-channel fused feature images corresponding to all collected labeled sound samples are randomly divided into training, validation, and test sets according to a certain ratio (70% training set, 15% validation set, 15% test set). The training set is used for model parameter learning, the validation set is used to adjust hyperparameters and monitor the model training process (to prevent overfitting), and the test set is used to finally evaluate the model's generalization ability.

[0078] Feature map level data augmentation: By randomly occluding or cropping images to form frequency or temporal masks (rotation or mirroring cannot be used because it will change the semantics of the sound), the size and diversity of the training dataset are expanded, thereby improving the generalization ability of the model.

[0079] Audio-level data augmentation: Before feature extraction, the original audio segments are augmented by randomly changing the volume, superimposing background noise collected from the target working conditions, time clipping, and random shifting.

[0080] Training parameter configuration: --in-chans 3: Specifies that the number of channels in the model input image is 3. This corresponds perfectly to the design in this invention that fuses three acoustic feature maps into a three-channel image (R, G, B).

[0081] --batch-size 24: Batch size, i.e., the number of samples input to the model in each iteration (parameter update). A larger batch size can provide more stable gradient estimates, but will increase memory consumption.

[0082] --lr 0.1: Initial learning rate. The learning rate controls the step size for updating model parameters. This is a relatively high initial learning rate, usually used in conjunction with learning rate warm-up and decay strategies.

[0083] --warmup-epochs 5: Number of warm-up epochs. During the first 5 epochs of training, the learning rate is gradually increased linearly from a very small value (e.g., 0 or close to 0) to the set initial learning rate of 0.1. This helps the model converge more stably in the early stages of training.

[0084] --epochs 300: The total number of epochs for model training. The entire training dataset will be traversed 300 times.

[0085] --min-lr 1e-6: Minimum learning rate. When using a learning rate scheduler (such as cosine annealing), the learning rate gradually decreases. This parameter sets a lower limit for the learning rate to prevent training from stalling due to an excessively small learning rate.

[0086] --sched cosine: Learning Rate Scheduler. Here, 'cosine' is selected, which is the cosine annealing scheduler. Under this strategy, the learning rate smoothly decays from the initial learning rate to the minimum learning rate or 0, following the shape of a cosine function.

[0087] `--lr-base-size 24`: The baseline batch size for learning rate adjustment. In some distributed training or specific learning rate adjustment strategies (such as linear scaling rules), if the actual `batch_size` differs from this `lr_base_size`, the initial learning rate may be adjusted accordingly (e.g., `actual_lr = lr * (current_batch_size / lr_base_size)`). In this embodiment, the user-defined `--batch-size 24` is the same as `--lr-base-size 24`, so the initial learning rate is directly set to `--lr 0.1`.

[0088] Loss function: For multi-class classification tasks, the cross-entropy loss function is typically used. This loss function measures the difference between the probability distribution predicted by the model and the one-hot encoding of the true labels.

[0089] Optimizer: Select the default Adam optimizer.

[0090] Model Evaluation: During training, the model's performance is evaluated on the validation set after each epoch, monitoring metrics such as loss, accuracy, precision, recall, and F1 score. The best model is selected based on its performance on the validation set (either the model with the lowest validation loss or the highest validation accuracy) and used to prevent overfitting (using an early stopping strategy). After training, the final selected model is comprehensively evaluated on an independent test set to test its generalization ability.

[0091] Reasoning process: Acquire a new 3-second audio signal of the idler roller running, to be monitored.

[0092] Following the steps of the aforementioned acoustic feature extraction module (extracting log-Mel spectrum, MFCC image, and spectral contrast image) and multi-feature fusion module (normalization, resizing, and three-channel overlay), the sound segment is converted into a 640x640x3 fused feature image.

[0093] Input this fused feature image into the already trained and loaded ResNet18 model.

[0094] The model propagates forward and outputs a probability vector containing 11 elements (corresponding to 11 state categories).

[0095] The category corresponding to the element with the highest probability value is selected as the fault diagnosis result for that sound segment.

[0096] An alarm threshold can be set. If the predicted probability of a specific fault category exceeds the preset threshold (0.8), the corresponding alarm signal will be triggered to alert maintenance personnel.

Claims

1. A method for monitoring idler roller faults by fusing multiple sound features, characterized in that, Includes the following steps: 1) Collect the sound signal during the operation of the idler roller, and divide the sound signal into segments of fixed duration to obtain multiple sound segments; 2) Extract three acoustic feature maps for each sound segment: log-Mel spectrogram, Mel frequency cepstral coefficient map, and spectral contrast map; 3) Normalize the three acoustic feature maps of each sound segment and adjust them to two-dimensional images of the same resolution to obtain normalized logarithmic Mel spectrogram, normalized Mel frequency cepstral coefficient map, and normalized spectral contrast map. 4) The normalized logarithmic Mel spectrum, the normalized Mel frequency cepstral coefficients, and the normalized spectral contrast diagram are used as the R, G, and B channels of the three-channel image, respectively, and fused to construct an acoustic feature color image. 5) Input the acoustic feature color image into the convolutional neural network model, identify the acoustic feature color image through the convolutional neural network model, and output the roller fault classification result corresponding to the sound segment.

2. The method for monitoring idler roller faults by sound multi-feature fusion according to claim 1, characterized in that, In step 3), the normalization process employs a minimum-maximum normalization method, and its normalization process can be described as follows: ； In the above formula, X norm It is the result of normalization; X is the pixel value of the original feature map; min(X) is the minimum value of the pixels in the original feature map; max(X) is the maximum value of the pixels in the original feature map.

3. The method for monitoring idler roller faults by sound multi-feature fusion according to claim 1 or 2, characterized in that, In step 2), the extraction process of the log-Mel spectrum includes the following steps: 21) The short-time Fourier transform of the original sound signal can be described by the following formula: ； In the above formula, x(t) is the original continuous-time signal, w(t−m) is a window function centered at m, ω is the frequency variable, and m is the position of the time sliding window; 22) Calculate the power spectrum for each frame, which can be described by the formula: ； In the above formula, k is the frequency index, m is the time frame index, X(k,m) represents the complex amplitude of the k-th frequency point of the m-th frame obtained by the short-time Fourier transform, and P(k,m) represents the power spectrum. 23) By weighting and summing the power spectrum using the Mel filter bank, the Mel spectrum is obtained as follows: ； In the above formula, S(m,t) represents the m-th Mel-band energy of the t-th frame, m is the Mel filter index, k is the frequency index, t is the time index, N is the number of short-time Fourier transform frequency points, and H... m (k) is the weighting coefficient of the m-th Mel filter at frequency point k; 24) Taking the logarithm of the Mel spectrum yields the logarithmic Mel spectrum: ； In the above formula, LogMel(m,t) is the logarithmic energy of the m-th Mel band in the t-th frame, and ϵ is a very small constant to prevent log(0); The extraction process of the Mel frequency cepstral coefficient map includes: 25) Perform Discrete Cosine Transform (DCT) on each frame based on the log-Mel spectrum to obtain the cepstral coefficients. The calculation formula is as follows: ； In the above formula, M is the number of Mel filters, m is the number of extracted coefficients, and MFCC(n,t) is the nth Mel frequency cepstral coefficient of the tth frame. The contrast in the spectral contrast diagram can be described as: ； In the above formula, B b Let C represent the set of frequency indices for the b-th frequency band, where ϵ is a very small constant to prevent division by zero. b (t) is the spectral contrast of the b-th frequency band at time frame t.

4. The method for monitoring idler roller faults by sound multi-feature fusion according to claim 1 or 2, characterized in that... In step 5), the convolutional neural network is a ResNet18 convolutional neural network.

5. The method for monitoring idler roller faults by sound multi-feature fusion according to claim 4, characterized in that, The ResNet18 convolutional neural network is constructed based on: an initial convolutional layer and a max pooling layer; four stages of residual blocks, each stage containing two residual blocks, each residual block consisting of two convolutional layers; and each convolutional layer followed by a normalization layer and a ReLU activation function. The system consists of a global average pooling layer, a fully connected layer as a classifier, followed by a Softmax activation function to output the probability distribution of each class. The training parameters for the neural network include: 3 input channels, a batch size of 24, an initial learning rate of 0.1, a minimum learning rate of 1e-6, 300 training epochs, cosine annealing as the learning rate scheduling strategy, and 5 warm-up epochs. The output includes 11 fault categories: motor noise, damaged roller coating, worn bearing seals, overly tight cleaner, material adhering to the idler roller skin, wear and noise from the idler roller shell, idler roller noise, worn idler roller bearing balls, damaged idler roller bearing, dust ingress into the bearing, and normal operation. The ReLU activation function is defined as follows: ； In the above formula, x represents the input value, and ReLU represents the output value; The Softmax activation function is defined as follows: ； In the above formula, Softmax(z) i ) is the output value of the Softmax function for the i-th input, representing the probability of that class, z. i It is the i-th component of vector z, z j Let be the j-th component of z, where j is the summation index variable and K is the number of categories.

Images