Bearing fault diagnosis method and device based on vibration-sound multi-channel fusion and medium
The bearing fault diagnosis method based on vibration and acoustic multimodal channel fusion utilizes discrete wavelet packet convolutional layers and channel attention mechanisms to construct an end-to-end bearing fault diagnosis model, solving the problems of insufficient information and high training costs in existing technologies, and achieving high-precision real-time fault diagnosis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWEST UNIV
- Filing Date
- 2026-03-17
- Publication Date
- 2026-06-05
Smart Images

Figure CN122153733A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a bearing fault diagnosis method for early fault type identification of key components in various electromechanical equipment, ensuring the safety and sustainability of production activities. Background Technology
[0002] With the continuous development of my country's economy, the widespread application of machinery and equipment in industrial production plays a crucial role in the stable operation of the social economy. Bearings, as core components of rotating machinery, directly affect the normal operation of equipment and production safety. However, most current bearing diagnostic methods are based on single-modal vibration sensing data. Faced with complex production environments, single-modal diagnostic methods often suffer from insufficient information, making it difficult to comprehensively capture complex fault characteristics. Although multimodal acoustic and vibration combined schemes can compensate for the limitations of single-modal data, existing methods still have some problems: cross-fusion at the data level may disrupt data continuity and introduce noise; fusion at the feature level can easily lead to feature redundancy and decreased accuracy; and model parallel schemes face challenges such as large parameter quantities and high training costs. Therefore, fully utilizing the advantages of various modal data and improving fault information for real-time intelligent diagnosis is of great significance for ensuring equipment reliability, improving production efficiency, and reducing safety risks. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide an end-to-end high-precision intelligent real-time fault diagnosis method, device and medium that can combine the advantages of vibration and acoustic multimodal data.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: This invention first provides a bearing fault diagnosis method based on the fusion of multi-modal vibration and acoustic channels, characterized by comprising: A bearing fault diagnosis model is constructed, which includes a discrete wavelet packet convolutional layer, a channel attention mechanism module, and a one-dimensional convolutional network. The discrete wavelet packet convolutional layer decomposes the input data into multi-channel frequency sub-bands. The channel attention mechanism module assigns weights to the multi-channel frequency sub-bands after decomposition by the discrete wavelet packet convolutional layer, assigning different weight ratios to different frequency sub-bands of different modes according to their importance to the diagnosis. The one-dimensional convolutional network extracts the deep features of the multi-channel frequency sub-bands after weight assignment by the channel attention mechanism module and obtains the classification results. A dataset including vibration and acoustic signals is constructed, and the dataset is divided into a training set and a test set. The constructed bearing fault diagnosis model is trained and tested using the training set and the test set to obtain a trained bearing fault diagnosis model. A trained bearing fault diagnosis model is used to diagnose faults based on fault data collected by sensors in real time.
[0005] This invention designs a discrete wavelet packet convolutional layer through a one-dimensional recursive convolutional layer to realize discrete wavelet packet transformation within the model; introduces an attention mechanism to assign weights to the frequency bands decomposed by the discrete wavelet packet convolutional layer; and then designs a one-dimensional convolutional network for further information mining.
[0006] The present invention also provides a bearing fault diagnosis device based on the fusion of vibration and acoustic multimodal channels, comprising a processor and a memory; the memory stores a program or instructions, which are loaded and executed by the processor to implement the steps of the bearing fault diagnosis method based on the fusion of vibration and acoustic multimodal channels.
[0007] The present invention also provides a computer-readable storage medium storing a program or instructions, wherein the program or instructions, when executed by a processor, implement the steps of the bearing fault diagnosis method of vibration and acoustic multimodal channel fusion.
[0008] Compared with the prior art, the beneficial effects of the present invention are: Compared to traditional fault diagnosis methods, the vibration and acoustic channel fusion convolutional neural network (VACF-CNN) diagnostic method provided in this invention has advantages such as eliminating the need for manual feature extraction and not being limited by prior knowledge. Compared to other multimodal intelligent diagnostic methods, VACF-CNN has higher diagnostic accuracy and faster training time, and also exhibits good robustness and generalization. Most existing multimodal intelligent fault diagnosis methods employ a model-parallel approach, but due to the high parameters of the model and cumbersome preprocessing limitations, the training time is long. Using a feature fusion approach results in feature redundancy in the classification layer, significantly reducing diagnostic accuracy. Furthermore, using a data fusion approach disrupts data coherence, introducing unnecessary noise and missing fault information.
[0009] The vibration and acoustic channel fusion convolutional neural network (VACF-CNN) diagnostic method provided by this invention has a simple operation for fusing acoustic and vibration signals without destroying the original continuity of the data; it eliminates the need for cumbersome one-dimensional to two-dimensional time-frequency preprocessing operations, and can realize end-to-end data input and output; it uses one-dimensional convolution and channel attention mechanism (SENet) to replace signal decomposition and filtering, and highly integrates signal analysis technology into the convolutional neural network, enhancing the interpretability of the model; it also requires fewer model parameters and has faster training and diagnostic speeds. Attached Figure Description
[0010] Figure 1 A flowchart for real-time intelligent fault diagnosis; Figure 2 This is an experimental system for actual data collection. Figure 3Schematic diagram of the convolutional neural network principle for fusion of vibration and acoustic channels; Figure 4 This is a comparison diagram of convolution in convolutional neural networks and the mathematical form of convolution. Figure 5 This is a schematic diagram of the channel attention mechanism. Figure 6 This is a schematic diagram of a convolutional neural network. Figure 7 This is a schematic diagram of t-SNE feature clustering; Figure 8 This is a schematic diagram of a confusion matrix; Figure 9 This is a schematic diagram of the noise reduction performance test. Figure 10 This is a schematic diagram of performance testing under varying operating conditions. Detailed Implementation
[0011] The present invention will be described in detail below with reference to the accompanying drawings, but this should not be construed as limiting the scope of protection of the present invention.
[0012] Example 1 like Figure 1 As shown, this embodiment provides a bearing fault diagnosis method based on convolutional neural network and vibration-acoustic multimodal channel fusion. This diagnostic method employs a vibration-acoustic channel fusion convolutional neural network (VACF-CNN) algorithm model. The vibration-acoustic channel fusion convolutional neural network (VACF-CNN) algorithm model includes: The dataset module, used to verify the performance of the algorithm model, is obtained by a specific device through vibration and acoustic sensors.
[0013] The data preprocessing module is used to ensure that the format of the offline training data and the data requiring real-time diagnosis meets the requirements of the model input before the data is input into the model.
[0014] The Discrete Wavelet Packet Convolutional Layer (WPT-Layer) module is used to perform time-frequency decomposition on the fused data of vibration and acoustic multi-mode channels. It is implemented by a one-dimensional recursive convolutional layer, and a series of frequency sub-bands of vibration and acoustic modes can be obtained after passing through this module.
[0015] The Channel Attention Mechanism (SENet) module is used to assign weights to the multi-channel frequency sub-bands after discrete wavelet packet convolutional layer decomposition. Different weight ratios are assigned to different frequency sub-bands of different modes according to their importance to the diagnosis, thereby reducing interference from redundant information.
[0016] The One-Dimensional Convolutional Neural Network (1DCNN) module is used for in-depth mining of fault information to uncover hidden deep features in the data. The linear layers within it are used for further integration and classification of these deep features.
[0017] The model training module is used to update and iterate the model weight parameters. It uses a set loss function and optimization algorithm to perform forward and backward propagation of the network to minimize the test set loss.
[0018] The visualization module is used to visualize the internal mechanisms of the algorithm model and quantify the specific classification.
[0019] The dataset module contains two datasets: field-collected data and publicly available online data. The field-collected data was acquired using a specific experimental setup via fiber optic vibration and acoustic sensors, and includes five data state types: bearing ball detachment fault, bearing ball splitting fault, gear tooth breakage fault, gear wear fault, and normal operation. The publicly available online data is the MAFAULDA dataset, which includes four data state types: outer rail fault, inner rail fault, rolling element fault, and normal operation. Each fault data type has five different load conditions: 0g, 6g, 20g, and 35g, and includes vibration and acoustic sensor acquisition channels.
[0020] The data preprocessing module includes, but is not limited to: handling missing values to prevent interference from outlier data, data normalization to improve model training speed, sliding window overlap segmentation for data augmentation, and constructing a dataset to divide the training and test sets. Finally, the preprocessed data from the two modalities are overlaid to complete multimodal data fusion.
[0021] The principle of the Discrete Wavelet Packet Convolutional Layer (WPT-Layer) module lies in the fact that it can achieve a mathematical convolution operation by reversing the kernel weights of the convolutional layer in a convolutional neural network. To ensure that the overall data length is halved after each pass through the Discrete Wavelet Packet Convolutional Layer (WPT-Layer) as in the wavelet packet transform (WPT) downsampling, the parameters need to be selected according to the following convolution size formula:
[0022] Where Output_size is the output data length, N is the input data length, F is the kernel size, P is the padding size, and S is the convolution stride. After multiple experiments, the initial wavelet filter coefficients were selected as "db4", the kernel size as 8, and the padding parameters as: cyclic padding with a padding size of 3.
[0023] The Channel Attention Mechanism (SENet) module mainly consists of three operations: Squeeze, Excitation, and Reweight. Squeeze performs global average pooling on each channel, compressing the spatial information of each channel into a scalar to obtain a global description of each channel and extract its importance features. Excitation generates a feature weight for each feature channel. Reweight multiplies the weights of each channel with the original channel's features to calculate the weighted output features.
[0024] The 1DCNN module consists of three convolutional layers, a global average pooling layer, and linear layers. Each convolutional layer includes a 1D convolutional layer, a 1D batch normalization (BatchNorm1d) layer, a ReLU activation function, and a max pooling layer. The convolutional layer parameter selection follows LeNet-5, using a combination of (kernel size, padding) = (3,1), which maintains the signal length and allows for the construction of deeper networks. It contains two linear layers. The first linear layer uses ReLU activation to integrate categorical features. The second linear layer is a classification layer, with softmax activation.
[0025] The model training module uses the Adaptive Moment Estimation (Adam) algorithm, which automatically adjusts the learning rate. The initial learning rate is set to 0.0001 to ensure a smoother training curve. The loss function is the multi-class cross-entropy loss function. The training method is mini-batch gradient descent, with the batch size set to 256 data points based on the computer's memory.
[0026] The visualization module includes t-random neighbor embedding (t-SNE) dimensionality reduction technology and a confusion matrix. t-SNE is used to reduce the dimensionality of classification layer features to two dimensions, allowing observation of planar clustering and providing insights into the model's internal mechanisms. The confusion matrix compares the number of real instances in a specific class with the number of predicted instances, showcasing the algorithm's specific classification performance.
[0027] The bearing fault diagnosis method based on convolutional neural networks and multimodal vibration and acoustic fusion generally consists of two parts: offline data training and real-time data diagnosis. Offline data training includes: 1. Data Acquisition: The actual acquired dataset comes from a specific fault monitoring experimental system based on fiber optic vibration sensors and acoustic sensors. It includes four fault states and a normal state. The experimental system is as follows: Figure 2As shown, an accelerometer is mounted above the motor housing to collect vibration data, and a fiber optic acoustic sensor is mounted 20 cm from the gearbox to monitor acoustic signals at a sampling frequency of 10 kHz. The motor speed is 1435 rpm. A monitoring system developed by an upper-level computer is responsible for collecting and processing fault signals.
[0028] 2. Data preprocessing involves performing preprocessing operations on the data collected by the multimodal sensors, including but not limited to handling missing values to prevent interference from anomalous data, channel fusion to supplement fault information, data normalization to improve model training speed, sliding window overlap segmentation for data augmentation, and constructing datasets to divide the training and test sets. Specifically, for self-built sets and datasets, overlapping sampling is used to increase the number of samples to prevent overfitting due to insufficient sample size.
[0029] 3. Modeling and training iterations of the VACF-CNN (Voice Channel Fusion Convolutional Neural Network) model, the process is as follows: Figure 3 As shown, the proposed algorithm model mainly consists of a discrete wavelet packet convolutional layer (WPT-Layer), a channel attention mechanism (SENet), and a one-dimensional convolutional network (1DCNN). Figure 4 As shown, the implementation of the discrete wavelet packet convolutional layer is achieved by assigning wavelet filter coefficients as weights to the convolution kernel and reversing their order. A single discrete wavelet packet convolutional layer decomposes the data into high-frequency and low-frequency sub-bands. This allows signal decomposition techniques to be introduced into convolutional neural networks, eliminating the need for additional time-frequency preprocessing for fault signal time-frequency analysis, thus achieving end-to-end data input and output. Furthermore, the initial wavelet filter coefficients are iteratively updated based on subsequent model training, solving the problem of accurate wavelet basis selection in traditional wavelet analysis. Then, the obtained series of frequency sub-bands with different modalities are fused, and a channel attention mechanism (SENet) is introduced. Figure 5As shown, the Channel Attention Mechanism (SENet) comprises three steps: Squeeze, Excitation, and Reweight. Each step achieves its goal through specific operations to enhance the model's feature selection ability. The specific steps are explained below: Squeeze: Through a global average pooling operation, the spatial information of each channel is compressed into a global feature vector. This operation generates a single scalar for each channel, representing the channel's global information. In this way, the Squeeze operation can effectively capture the global dependencies of each channel. Excitation: In the excitation phase, the Channel Attention Mechanism (SENet) generates weights for each channel through a two-layer fully connected network. The first layer uses the ReLU activation function to capture the non-linear relationships between channels, and the second layer uses the Sigmoid function to generate normalized weights representing the relative importance of each channel. The goal of Excitation is to enable the model to adaptively assign higher weights to important channels. Reweight: Finally, through a channel-by-channel weighting operation, the weights generated by Excitation are applied to each channel. This process is equivalent to assigning a weight factor to each channel, making the model focus more on channels important for classification, thereby improving the expressive power of features. After the channel attention mechanism (SENet) reweights all channels, i.e., all modal frequency subbands, the frequency subbands of the modal that are more helpful for classification will be highlighted, while interference information will be suppressed, effectively performing a "noise reduction" operation inside the convolutional neural network. Then, this multi-channel data is fed into a pre-designed one-dimensional convolutional neural network (1DCNN) for deep feature extraction, such as... Figure 6 As shown, a Convolutional Neural Network (CNN) mainly consists of convolutional layers, pooling layers, and fully connected layers. In the convolutional layers, multiple convolutional kernels are convolved with the input image, and after adding bias, an activation function is applied to obtain a series of feature maps. Pooling layers are often placed after convolutional layers to reduce the dimensionality of the feature maps. In the pooling layers, pooling operations are performed on each non-overlapping n×n region of the feature map output by the convolutional layer, selecting the maximum or average value in each region, ultimately reducing the output image by a factor of n in both dimensions. After the input image passes through multiple convolutional and pooling layers, the fully connected network classifies the extracted features. In the fully connected layers, the input is a weighted sum of the one-dimensional feature vectors expanded from all feature maps, passed through an activation function. After passing through a Convolutional Neural Network (CNN), the data is highly condensed into a series of deep features. The overall model parameters established based on the actual dataset are shown in Table 1.
[0030] Table 1
[0031] After modeling, the model was trained using stochastic mini-batch gradient descent. The dataset was divided into multiple mini-batches, and gradient calculation and parameter updates were performed on each mini-batch. Compared to traditional full gradient descent, this method reduces computational overhead and, by using only a portion of the data each time, can escape local optima to some extent, resulting in more stable convergence. Due to computer memory limitations, a batch size of 256 was chosen. The loss function and optimization algorithm used were the multi-class cross-entropy function and the Adaptive Moment Estimation (Adam) algorithm, respectively. An initial learning rate of 0.0001 was set to make the training curve smoother. For the comparative model with more parameters and higher memory usage, a cutoff condition was set: early stopping was triggered if the loss did not decrease by a threshold (tol) for five consecutive iterations, where tol = 10**(-5). For the other models, the training epochs were uniformly set to 300, and the loss and accuracy on the test set after each epoch were plotted. This invention is compared with three mainstream algorithm models in the current field of intelligent bearing fault diagnosis: continuous wavelet transform-based two-dimensional convolutional neural network (CWT-2DCNN), one-dimensional convolutional neural network (1DCNN), and discrete wavelet packet convolutional neural network (WPT-CNN). These four algorithm models were trained on a self-built dataset, and the specific training results are shown in Table 2. Table 2
[0032] Table 2 shows that the VAF-CNN algorithm has higher diagnostic accuracy and faster training speed compared to the other three algorithms. It can achieve 100% diagnostic accuracy on the test set while only requiring an average of 1.51 seconds to train one training epoch.
[0033] 4. Visualization: To further demonstrate the internal mechanism of VCF-CNN and the specific classification results, t-SNE dimensionality reduction and a confusion matrix are introduced. By performing two-dimensional dimensionality reduction on the classification layer features, it becomes clear whether the features extracted by the algorithm for the classification task are reasonable. The original data distribution of the actual dataset and the clustering diagram of the classification layer features extracted by this invention are shown below. Figure 7As shown. A confusion matrix is a visualization tool used to evaluate the performance of a classification model. Each row of the matrix represents the actual class, and each column represents the predicted class. By observing the values on the diagonal, you can see the number of samples correctly classified by the model, while the values off-diagonal represent the number of samples misclassified by the model. The confusion matrix provides a concrete and quantitative understanding of the model's specific classification performance. The confusion matrix of the proposed invention on the actual collected dataset is shown below. Figure 8 As shown.
[0034] 5. Comparative Testing of Vibration-Acoustic Fusion Methods: To further verify the superiority of this invention, three different vibration-acoustic fusion methods were experimentally analyzed using the MAFAULDA public dataset. These three methods are data-level fusion, feature-level fusion, and the channel-level fusion proposed in this invention. The data-level vibration-acoustic fusion method involves cross-combining vibration and acoustic data to create new data. First, the data from different bearing modes are cross-processed, specifically, the first sampling point of the first mode is immediately followed by the first sampling point of the second mode, and so on. The feature-level fusion method employs a parallel model approach, importing vibration data and acoustic sensing data as separate datasets into the model for feature extraction. Finally, the extracted features are merged and fused at the classification layer. The structure of the vibration-acoustic channel fusion convolutional neural network (VACF-CNN) was appropriately adjusted, and these three methods were used for training. The specific training results are shown in Table 3. Table 3
[0035] According to Table 3, the channel-level fusion proposed in this invention has a higher classification accuracy.
[0036] 6. Noise Resistance and Variable Operating Condition Testing: To further verify the robustness and generalization of the proposed algorithm model, the Vibration and Acoustic Channel Fusion Convolutional Neural Network (VACF-CNN), Gaussian white noise with signal-to-noise ratios of 0dB, 10dB, and 20dB was added to the MAFAULDA public dataset for training, simulating strong noise, medium noise, and low noise environments, respectively. The changes in diagnostic accuracy before and after were compared. Specific noise resistance results are shown below. Figure 9 As shown in Table 4.
[0037] Table 4
[0038] Simultaneously, the test and training sets were re-divided according to the different loads in the original data, allowing the model to learn from data under three other load conditions and then test data under the 0g load condition to evaluate the classification performance of the proposed model under varying load conditions. Specific experimental results are as follows: Figure 10 As shown in Table 4.
[0039] The real-time data diagnostic process includes: Figure 2 As shown, after the model has been trained on the dataset for a certain number of rounds, the optimal model weights from the round with the highest accuracy during training are saved, indicating that training is complete. After new fault data is collected by sensors, the data is preprocessed, and then the saved weight file is loaded and imported into the trained model to obtain the diagnostic results.
[0040] Example 2 This embodiment provides a bearing fault diagnosis device based on the fusion of vibration and acoustic multimodal channels, including a processor and a memory; the memory stores a program or instructions, which are loaded and executed by the processor to implement the steps of the bearing fault diagnosis method based on the fusion of vibration and acoustic multimodal channels provided in Embodiment 1.
[0041] Example 3 This embodiment provides a computer-readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements the steps of the phase demodulation method based on a digital carrier provided in Embodiment 1.
Claims
1. A bearing fault diagnosis method based on multi-modal channel fusion of vibration and acoustics, characterized in that, include: A bearing fault diagnosis model is constructed, which includes a discrete wavelet packet convolutional layer, a channel attention mechanism module, and a one-dimensional convolutional network. The discrete wavelet packet convolutional layer decomposes the input data into multi-channel frequency sub-bands; the channel attention mechanism module assigns weights to the multi-channel frequency sub-bands after decomposition by the discrete wavelet packet convolutional layer, assigning different weight ratios according to the importance of different frequency sub-bands of different modalities to the diagnosis; the one-dimensional convolutional network extracts the deep features of the multi-channel frequency sub-bands after weight assignment by the channel attention mechanism module and obtains the classification results. A dataset including vibration and acoustic signals is constructed, and the dataset is divided into a training set and a test set. The constructed bearing fault diagnosis model is trained and tested using the training set and the test set to obtain a trained bearing fault diagnosis model. A trained bearing fault diagnosis model is used to diagnose faults based on fault data collected by sensors in real time.
2. The bearing fault diagnosis method based on multi-mode channel fusion of vibration and acoustics according to claim 1, characterized in that, The channel attention mechanism module includes compression, activation, and reweighting operations. The compression operation performs global average pooling on each channel, compressing the spatial information of each channel into a scalar to obtain a global description of each channel and extract the importance features of each channel. The activation operation generates a feature weight for each feature channel; the reweighting operation multiplies the weight of each channel with the original feature to calculate the weighted output feature.
3. The bearing fault diagnosis method based on multi-modal channel fusion of vibration and acoustics according to claim 2, characterized in that, During the activation process, a two-layer fully connected network is used to generate the weights for each channel. The first fully connected layer uses the ReLU activation function to capture the non-linear relationship between channels, and the second fully connected layer uses the Sigmoid function to generate normalized weights that represent the relative importance of each channel.
4. The bearing fault diagnosis method based on multi-modal channel fusion of vibration and acoustics according to claim 1, characterized in that, A one-dimensional convolutional network consists of convolutional layers, pooling layers, and fully connected layers. Deep feature extraction is performed in the convolutional layers, dimensionality reduction of the extracted features is performed in the pooling layers, and classification of the dimensionality-reduced features is performed in the fully connected layers.
5. The bearing fault diagnosis method based on multi-mode channel fusion of vibration and acoustics according to claim 4, characterized in that, The classification results output by the fully connected layer of the one-dimensional convolutional network are reduced to two dimensions using t-random neighbor embedding (t-SNE); the confusion matrix is used to compare and display the distribution of the number of actual fault categories and predicted fault categories.
6. The bearing fault diagnosis method based on multi-mode channel fusion of vibration and acoustics according to claim 1, characterized in that, The dataset includes actual field-collected data and publicly available online data. The actual field-collected data is obtained through fiber optic vibration sensors and acoustic sensors, and there are five data status types. The publicly available online data is the MAFAULDA dataset, which has four data status types, and each data type has five different load conditions, and has vibration sensor acquisition channels and acoustic sensor acquisition channels.
7. The bearing fault diagnosis method based on multi-mode channel fusion of vibration and acoustics according to claim 1, characterized in that, The parameters of the discrete wavelet packet convolutional layer are selected according to the following convolution size formula: Where Output_size is the output data length, N is the input data length, F is the kernel size, P is the padding size, and S is the stride.
8. The bearing fault diagnosis method based on multi-mode channel fusion of vibration and acoustics according to claim 7, characterized in that, The initial wavelet filter coefficients are set to "db4"; the kernel size F is set to 8; and the padding parameters are: cyclic padding and padding size P is 3.
9. A bearing fault diagnosis device based on multi-mode vibration and acoustic fusion, characterized in that, It includes a processor and a memory; the memory stores a program or instructions, which are loaded and executed by the processor to implement the steps of the bearing fault diagnosis method of vibration and acoustic multimodal channel fusion as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the bearing fault diagnosis method of vibration and acoustic multimodal channel fusion as described in any one of claims 1 to 8.