A bearing intelligent fault diagnosis method fusing acoustic and vibration signals

Abstract

The application belongs to the technical field of intelligent fault diagnosis and predictive maintenance of equipment, and particularly relates to a bearing intelligent fault diagnosis method fusing sound and vibration signals, characterized in that the method realizes rapid diagnosis of bearing faults based on analysis of the fused sound and vibration signals, and comprises the steps of collecting sound and vibration signals, extracting and fusing, enhancing feature processing, model training and generating a fault diagnosis result.The application has the beneficial effect that one-dimensional convolutional neural networks are used for feature extraction, and multi-modal attention (MMA) is used for feature fusion, so that the complementary advantages of sound signals and vibration signals are fully utilized.A K-Nearest Neighbor (KNN) graph is dynamically constructed based on the fused features, and a graph convolution network (GCN) is applied.A multi-task learning (MTL) framework is introduced, with fault classification as the main task and modal classification as the auxiliary task.The application can obtain more rich bearing fault feature information, has high classification accuracy, and provides an effective solution for bearing intelligent fault diagnosis in industrial applications.

Description

Technical Field

[0001] This invention belongs to the field of intelligent fault diagnosis and predictive maintenance technology for mechanical equipment, and particularly relates to an intelligent fault diagnosis method for bearings that integrates acoustic and vibration signals. Background Technology

[0002] With the continuous advancement of science and technology and industrialization, the modern machinery field is constantly evolving, moving towards higher precision, superior performance, and intelligence. Predictive health management (PHM) of mechanical systems is essential to avoid significant economic losses and personnel casualties for enterprises. The main purpose of PHM is to diagnose faults and predict the remaining life of mechanical systems based on historical and current condition monitoring data. Rolling bearings, as key components in rotating machinery, directly affect the reliability and safety of the entire equipment due to their operating condition. However, due to complex working environments and prolonged operation, rolling bearings are highly susceptible to wear, fatigue, and spalling, leading to decreased equipment performance and potentially causing serious safety accidents. Therefore, timely and accurate detection and diagnosis of rolling bearing faults are crucial for ensuring the normal operation of mechanical equipment.

[0003] Sound and vibration signals are complementary in terms of frequency range, sensitivity, and acquisition methods. Fusing these signals can provide more comprehensive fault information and improve the accuracy and reliability of diagnosis. Multimodal fusion combines data from different sources to obtain richer information and utilizes the correlation between physical quantities to improve diagnostic performance. Currently, some researchers have explored the fusion of sound and vibration signals for bearing fault diagnosis. However, most fusion methods rely on feature-level and decision-level fusion, depending on manually extracted features, and fail to fully mine deeper data information. Challenges such as spatiotemporal alignment of different modal data and noise interference increase the difficulty of fusion. Effectively integrating sound and vibration signals to fully mine deep multimodal data remains a challenging problem. Furthermore, for the temporal features of vibration and sound signals, traditional convolutional neural networks mainly focus on extracting features from individual samples, ignoring the correlation information between samples. This makes it difficult to fully utilize the potential structural information of multimodal data, thus limiting the improvement of model performance to some extent. Summary of the Invention

[0004] The purpose of this invention is to provide a bearing intelligent fault diagnosis method that integrates acoustic and vibration signals, overcoming the shortcomings of existing technologies. This method is used to diagnose the working status of bearings in complex industrial scenarios by integrating acoustic and vibration signals, efficiently utilizing bearing fault data resources, and providing a feasible means for intelligent bearing fault diagnosis.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A bearing intelligent fault diagnosis method based on fused acoustic and vibration signals achieves rapid bearing fault diagnosis based on the analysis of fused acoustic and vibration signals. The method includes acquiring acoustic and vibration signals, extracting and fusing them, enhancing feature processing, model training, and generating fault diagnosis results. Specifically, it includes the following steps:

[0007] 1) Acquire acoustic and vibration signals: acquire vibration signals generated during the operation of the rolling bearing through a vibration sensor, and acquire sound signals generated during the operation of the rolling bearing through a microphone array sensor;

[0008] 2) Extraction and fusion: CNN is used to extract deep features from the collected sound and vibration signals. Then, MMA is applied to fuse these features. Attention weights are calculated to highlight the most relevant features in each modality, thereby obtaining the fused feature representation.

[0009] 3) Enhanced feature processing: The fused feature vectors are used to construct a graph representation of the data, where nodes represent samples and edges are formed based on their similarity. The graph structure is created using the KNN algorithm. Then, the fused features and graph are passed to GCN. GCN captures the local and global relationships between samples, thereby generating an enhanced feature representation.

[0010] 4) Model training: Input the GCN output into an MTL for fault classification. The MTL includes a main task and auxiliary tasks. Define a total loss function and train the model by minimizing this loss.

[0011] 5) Generate fault diagnosis results, and use confusion matrix and classification scatter plot to intuitively evaluate the prediction effect of the model, and obtain accurate fault diagnosis results based on fused acoustic and vibration signals.

[0012] Furthermore, in the acquisition of acoustic and vibration signals in step 1), the sensor is arranged as follows: a vibration sensor is installed on the bearing seat of the bearing under test, and an acoustic array sensor consisting of 16 microphones is arranged on a plane 200 mm away from the end of the bearing under test, with the central axis of the array aligned with the axis of the bearing.

[0013] Furthermore, step 2) of the extraction and fusion process includes extraction, modality attention scoring, defining attention weights, and fusing feature vectors. The specific operations are as follows:

[0014] 11) Extraction: Sound and vibration signals are processed through multiple one-dimensional convolutions to extract local patterns. Then, pooling layers are used for downsampling, and finally, a fully connected layer projects the final output onto a low-dimensional feature space. The extraction process can be described mathematically as follows:

[0015]

[0016]

[0017] In the formula, This represents the sound signal value at the nth synchronization time point. Representing the vibration signal value at the same time point, Conv(•) represents the convolution operation, σ(•) is the activation function, Pool(•) represents the pooling operation, and the output is a d-dimensional feature vector;

[0018] 12) Modal attention scores: The modal attention mechanism calculates the attention scores sa and sv for acoustic and vibration modes respectively, reflecting the relative importance of each mode. The specific formula is as follows:

[0019]

[0020]

[0021] in, This represents the feature vector extracted from the sound signal. This represents the feature vector extracted from the vibration signal. It is a learnable weight matrix. For bias terms, This is a learnable attention vector;

[0022] 13) Define attention weights. After obtaining the attention scores for the two modes, apply softmax normalization to calculate the attention weights, which represent the relative importance of each mode during the fusion process. The attention weights for the acoustic and vibration modes are defined as follows:

[0023]

[0024]

[0025] In the formula, Sa and Sv are the attention scores for the acoustic mode and vibration mode, respectively;

[0026] 14) Fusing Feature Vectors: After calculating the attention weights for the acoustic and vibration modes, the features of the two modes are weighted and summed to obtain the final fused feature vector.

[0027]

[0028] in, It is a fusion of feature vectors. and The attention weight for each modality is applied to the acoustic and vibration feature vectors respectively. The fused feature vector hfused combines information from both modalities, reflecting the different importance of each modal feature and providing a representation for subsequent classification tasks.

[0029] Furthermore, step 3) enhances feature processing, which includes constructing a feature matrix, assigning neighbor weights, normalizing the adjacency matrix, and predicting node categories. The specific steps are as follows:

[0030] 21) Constructing the feature matrix: A K-Nearest Neighbor (KNN) graph was built using fused features. Nodes were connected based on the similarity between samples. The feature matrix H, composed of the fused feature vectors hfused, is represented as follows:

[0031]

[0032] Where N represents the total number of samples and d represents the feature dimension of each sample; in order to construct the KNN graph, Euclidean distance is used to measure the similarity between samples, thereby determining the connection between nodes. For each node, K nearest neighbors are selected, and an adjacency matrix is ​​constructed based on the similarity between samples. Each edge in the adjacency matrix represents the similarity between connected samples.

[0033] 22) Assign neighbor weights using a Graph Attention Network (GAT) with an attention mechanism. This network can assign different neighbor weights to each node. For node i, its aggregation features... Represented as:

[0034]

[0035] in, Let deg(i) represent the set of neighboring nodes of node i, and deg(j) represent the degrees of nodes i and j, respectively; W is the weight matrix shared by the entire layer.

[0036] 23) Normalized adjacency matrix. The normalized adjacency matrix is ​​defined as follows:

[0037]

[0038] In the formula, A is the adjacency matrix of the graph, D is the degree matrix, and ;

[0039] 24) Predicting Node Classes: After aggregating information from neighboring nodes, the output layer of GCN uses a classifier to predict the class of each node. The classification expression is as follows:

[0040]

[0041] In the formula, It is the weight matrix of the classifier. is the bias term of the classifier, and C is the number of classes.

[0042] Furthermore, the model training in step 4) includes modal task classification and determining the total task loss, and the specific operation steps are as follows:

[0043] 31) Modal task classification: In a multi-task learning framework, the class label yclass of each sample is predicted first to determine its class. The fused features hfused are used to identify whether the features come from acoustic or vibration modes. The modal classifier used to identify the source modes of the fused features is calculated as follows:

[0044]

[0045] in, It is the weight matrix of the modal classifier, which projects the fused features onto a two-dimensional probability space, representing acoustic and vibration modes; It is the bias term of the modality classifier. It is a fused feature vector that integrates acoustic and vibrational characteristics;

[0046] The actual label ymodality for each modality is defined as follows:

[0047] Regarding acoustic characteristics:

[0048]

[0049] Regarding vibration characteristics:

[0050]

[0051] Through modality classification tasks, the model can better distinguish features from different modalities;

[0052] 32) Determine the total task loss. In the multi-task learning process, the model's loss function consists of the main classification task loss and the auxiliary modality classification task loss. The final loss is the weighted sum of the main task loss and the auxiliary task loss.

[0053] The loss function for the main classification task is cross-entropy loss, which measures the difference between the predicted class label and the true class label, and is defined as follows:

[0054]

[0055] In the formula, N is the number of samples, and C is the number of categories. This represents the true label of sample i for category j. It is the predicted probability of sample i belonging to category j from the model;

[0056] The modality classification task is used to balance the model's accuracy in modality classification, and its loss function is defined as follows:

[0057]

[0058] In the formula, Let be the modal label for sample i. For acoustic features, the label is [1,0]; for vibration features, the label is [0,1]. It is the predicted probability of the model;

[0059] The total loss is a weighted sum of the classification loss and the modality classification loss, and is expressed as:

[0060]

[0061] In the formula, λclassification (classification) and λmodality (modality) are weight parameters.

[0062] Compared with the prior art, the beneficial effects of the present invention are:

[0063] 1) Feature Extraction and Modality Fusion: A one-dimensional convolutional neural network is used to extract features from the sound and vibration signals respectively to obtain high-level feature representations. A modality attention mechanism is introduced to automatically learn the weights of different modalities, achieving effective feature fusion and fully leveraging the complementary advantages of sound and vibration signals;

[0064] 2) Construction of K-Nearest Neighbor Graph and Application of Graph Neural Network: Based on fused features, a K-nearest neighbor graph is dynamically constructed between samples. Samples are treated as nodes in the graph, and information is propagated on the graph structure using a graph convolutional network (GCN). This method captures the potential correlations between samples, improving the accuracy of fault classification;

[0065] 3) Multi-task learning framework: A multi-task learning strategy is adopted, with fault classification as the main task and modality classification as an auxiliary task to classify the features of each modality. Through joint training, the information from the auxiliary task enhances the model's generalization ability and prevents overfitting. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating the process of an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of the experimental system structure according to an embodiment of the present invention;

[0068] Figure 3 The images show the time-domain signals of vibration and sound data for seven different fault types according to this invention. The left column represents the vibration signal, and the right column represents the sound signal.

[0069] Figure 4The results of comparing six different ablation methods in Example 1 of this invention;

[0070] Figure 5 The results of the classification tests of bearings using different methods in Embodiments 1-6 of this invention;

[0071] In the diagram: 1-motor, 2-test bearing, 3-vibration sensor, 4-acoustic array sensor, 5-data acquisition system. Detailed Implementation

[0072] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0073] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The components of the embodiments of the present invention described and shown in the accompanying drawings can typically be arranged and designed in many different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.

[0074] See Figure 1 This diagram illustrates the workflow of an embodiment of the intelligent bearing fault diagnosis method that fuses acoustic and vibration signals according to the present invention. The method proposes MTMAGNet (Multi-Task Multimodal Attention Graph Convolutional Network), which effectively fuses acoustic and vibration signals by combining multimodal attention (MMA), multi-task learning (MTL), and graph convolutional networks (GCN). First, a one-dimensional convolutional neural network is used for feature extraction, followed by feature fusion using MMA, fully leveraging the complementary advantages of acoustic and vibration signals to effectively fuse their information. To capture potential correlations between samples, a k-nearest neighbor (KNN) graph is dynamically constructed based on the fused features, and GCN is applied. Furthermore, an MTL framework is introduced, with fault classification as the primary task and modal classification as an auxiliary task, enhancing generalization ability and preventing overfitting. Finally, intelligent fault diagnosis of the bearing is achieved through the above methods.

[0075] This invention provides a bearing intelligent fault diagnosis method based on the fusion of acoustic and vibration signals. It achieves rapid bearing fault diagnosis through the analysis of fused acoustic and vibration signals, including acquiring acoustic and vibration signals, extracting and fusing them, enhancing feature processing, model training, and generating fault diagnosis results. Specifically, it includes the following steps:

[0076] 1) Acquire acoustic and vibration signals. Acquire vibration signals generated during the operation of the rolling bearing through a vibration sensor and sound signals generated during the operation of the rolling bearing through a microphone array sensor. The sensor arrangement is as follows: Install a vibration sensor on the bearing housing of the bearing under test, and arrange a sound array sensor consisting of 16 microphones on a plane 200 mm away from the end of the bearing under test, and align the central axis of the array with the axis of the bearing.

[0077] 2) Extraction and Fusion: A CNN is used to extract deep features from the acquired sound and vibration signals. Then, MMA is applied to fuse these features. Attention weights are calculated to highlight the most relevant features in each modality, resulting in a fused feature representation. Extraction and fusion include steps such as extraction, modality attention scoring, defining attention weights, and fusing feature vectors. The specific operations are as follows:

[0078] 11) Extraction: Sound and vibration signals are processed through multiple one-dimensional convolutions to extract local patterns. Then, pooling layers are used for downsampling, and finally, a fully connected layer projects the final output onto a low-dimensional feature space. The extraction process can be described mathematically as follows:

[0079] Formula (1)

[0080] Formula (2)

[0081] In the formula, This represents the sound signal value at the nth synchronization time point. Representing the vibration signal values ​​at the same time point, Conv(•) denotes the convolution operation, σ(•) is the activation function, and Pool(•) denotes the pooling operation. The output is a d-dimensional feature vector. Through convolution, the model can capture local signal patterns, which helps reduce the influence of noise. Pooling layers help reduce data dimensionality, avoiding overfitting while retaining key information. By passing through multiple convolutional and pooling layers, the original time-series signal is compressed into a low-dimensional feature vector, which can be more effectively fused in subsequent multimodal fusion and classification tasks.

[0082] 12) Modal attention scores: The modal attention mechanism calculates the attention scores sa and sv for acoustic and vibration modes respectively, reflecting the relative importance of each mode. The specific formula is as follows:

[0083] Formula (3)

[0084] Formula (4)

[0085] in, This represents the feature vector extracted from the sound signal. This represents the feature vector extracted from the vibration signal. It is a learnable weight matrix. For bias terms, This is a learnable attention vector;

[0086] 13) Define attention weights. After obtaining the attention scores for the two modes, apply softmax normalization to calculate the attention weights, which represent the relative importance of each mode during the fusion process. The attention weights for the acoustic and vibration modes are defined as follows:

[0087] Formula (5)

[0088] Formula (6)

[0089] In the formula, Sa and Sv are the attention scores of the acoustic mode and the vibration mode, respectively. It can be observed from equations (5) and (6) that the sum of the attention weights of the two modes is equal to 1, ensuring that the modes are fused in a weighted manner.

[0090] 14) Fusing Feature Vectors: After calculating the attention weights for the acoustic and vibration modes, the features of the two modes are weighted and summed to obtain the final fused feature vector.

[0091] Formula (7)

[0092] in, It is a fusion of feature vectors. and The attention weight for each modality is applied to the acoustic and vibration feature vectors respectively. The fused feature vector hfused combines information from both modalities, reflecting the different importance of each modal feature and providing rich representations for subsequent classification tasks.

[0093] 3) Enhanced feature processing: A graph representation of the data is constructed using the fused feature vectors, where nodes represent samples and edges are formed based on their similarity. The graph structure is created using the KNN algorithm. Then, the fused features and graph are passed to the GCN, which captures the local and global relationships between samples, thereby generating an enhanced feature representation. Step 3) Enhanced feature processing includes constructing a feature matrix, assigning neighbor weights, normalizing the adjacency matrix, and predicting node categories. The specific steps are as follows:

[0094] 21) Constructing the feature matrix: A K-Nearest Neighbor (KNN) graph was built using fused features. Nodes were connected based on the similarity between samples. The feature matrix H, composed of the fused feature vectors hfused, is represented as follows:

[0095] Formula (8)

[0096] Where N represents the total number of samples and d represents the feature dimension of each sample; in order to construct the KNN graph, Euclidean distance is used to measure the similarity between samples, thereby determining the connection between nodes. For each node, K nearest neighbors are selected, and an adjacency matrix is ​​constructed based on the similarity between samples. Each edge in the adjacency matrix represents the similarity between connected samples.

[0097] 22) Assign neighbor weights using a Graph Attention Network (GAT) with an attention mechanism. This network can assign different neighbor weights to each node. For node i, its aggregation features... Represented as:

[0098] Formula (9)

[0099] in, Let deg(i) represent the set of neighboring nodes of node i, and deg(j) represent the degrees of nodes i and j, respectively; W is the weight matrix shared by the entire layer.

[0100] 23) Normalized adjacency matrix. The normalized adjacency matrix is ​​defined as follows:

[0101] Formula (10)

[0102] In the formula, A is the adjacency matrix of the graph, D is the degree matrix, and ;

[0103] 24) Predicting Node Classes: After aggregating information from neighboring nodes, the output layer of GCN uses a classifier to predict the class of each node. The classification expression is as follows:

[0104] Formula (11)

[0105] In the formula, It is the weight matrix of the classifier. is the bias term of the classifier, and C is the number of classes.

[0106] 4) Model training: The GCN output is input into an MTL for fault classification. This MTL includes a primary task and auxiliary tasks. A total loss function is defined, and the model is trained by minimizing this loss. Step 4) of model training includes modal task classification and determining the total task loss. The specific steps are as follows:

[0107] 31) Modal task classification: In a multi-task learning framework, the model needs to learn both the primary and auxiliary tasks simultaneously. The primary task is to predict the class label yclass for each sample, determining its class. On the other hand, the auxiliary task utilizes the fused features h... fused This task helps identify whether features originate from acoustic or vibrational modes. This auxiliary task enhances the model's sensitivity to inter-modal differences, thereby improving the effectiveness of multimodal feature fusion. The modal classifier used to identify the source modes for fusion features is calculated as follows:

[0108] Formula (12)

[0109] in, It is the weight matrix of the modal classifier, which projects the fused features onto a two-dimensional probability space, representing acoustic and vibration modes; It is the bias term of the modality classifier. It is a fused feature vector that integrates acoustic and vibrational characteristics;

[0110] To train the modality classifier, the ground truth label ymodality for each modality is encoded as follows:

[0111] Regarding acoustic characteristics:

[0112] Formula (13)

[0113] Regarding vibration characteristics:

[0114] Formula (14)

[0115] Through modality classification tasks, the model can better distinguish features from different modalities, and this supervised learning helps improve the quality of fused features.

[0116] 32) Determine the total task loss. In multi-task learning, the model's loss function consists of the main classification task loss and the auxiliary modality classification task loss. To ensure optimal performance on both tasks, the final loss is a weighted sum of the main task loss and the auxiliary task loss.

[0117] The loss function for the main classification task is cross-entropy loss, which measures the difference between the predicted class label and the true class label, and is defined as follows:

[0118] Formula (15)

[0119] In the formula, N is the number of samples, and C is the number of categories. This represents the true label of sample i for category j. It is the predicted probability of sample i belonging to category j from the model;

[0120] The modality classification task is used to balance the model's accuracy in modality classification, and its loss function is defined as follows:

[0121] Formula (16)

[0122] In the formula, Let be the modal label for sample i. For acoustic features, the label is [1,0]; for vibration features, the label is [0,1]. It is the predicted probability of the model;

[0123] The total loss is a weighted sum of the classification loss and the modality classification loss, and is expressed as:

[0124] Formula (17)

[0125] In the formula, where λ classification Classification and λ modality The modalities are weight parameters. These weights adjust the importance of the primary and secondary tasks in the total loss function.

[0126] 5) Generate fault diagnosis results, and use confusion matrix and classification scatter plot to intuitively evaluate the prediction effect of the model, and obtain accurate fault diagnosis results based on fused acoustic and vibration signals.

[0127] The test system of this invention is shown in the embodiment. Figure 2 The system includes a motor 1, a test bearing 2, a vibration sensor 3, an acoustic array sensor 4, and a data acquisition system 5. The acoustic array sensor 4 consists of 16 BSWA MPA416 acoustic sensors, each with a sensitivity of 50 mV / Pa. These sensors are arranged in a circular array mounted on a disk to form an acoustic array plate. To ensure signal quality and reduce signal attenuation, the acoustic array plate is mounted on a plane 200 mm from the end of the bearing under test, with its center aligned with the bearing axis.

[0128] During bearing operation, an acoustic array sensor collects acoustic signals and transmits them to a computer terminal via a PAK MKII-SC42 data acquisition system, converting the analog signals into digital signals. To meet the requirements of the sampling theorem, the sampling frequency of the acoustic signals is set to 16384Hz. A vibration sensor is installed on the bearing housing of the bearing under test to collect vibration signals. These signals are then acquired using a BSZ800D-16 data acquisition unit, also with a sampling frequency set to 16384Hz. The vibration data acquisition device is connected to the computer terminal and used in conjunction with the acoustic signals for subsequent data analysis.

[0129] Figure 3The temporal signals of vibration and acoustic data collected for seven different fault types are presented. Each subplot shows the amplitude variation over time of the vibration and acoustic records under specific fault conditions. In the left column of each plot, the vibration signals exhibit extensive amplitude fluctuations under various fault conditions, indicating potential differences in energy distribution and fault impact intensity. The right column shows the acoustic signals, whose amplitude variations are consistent with the fault characteristics described by the vibration signals. These signals provide fundamental insights into the characteristics of each fault type and will be used in the feature extraction and classification stages of the proposed multimodal diagnostic framework.

[0130] The rolling bearings in the test system operated at 2400 rpm. Considering different fault types, seven different operating conditions were constructed for the rolling bearings, including two normal operating conditions and six fault conditions: outer ring fault, inner ring fault, rolling element fault, and partial coupling fault. Each condition was assigned a label from C0 to C6 to represent a different load state. For both acoustic and vibration signals, 400 samples were collected for each condition, totaling 2800 samples for each signal type, with 20% used for validation.

[0131] The feature extraction module consists of three one-dimensional convolutional layers with kernel sizes of 5, 5, and 3, respectively. The input sizes increase sequentially to 1, 64, and 128, resulting in output sizes of 64, 128, and 256. A 0.5 Dropout layer is added after each convolutional layer to prevent overfitting, followed by a 128-sized fully connected layer. The modality attention module comprises three linear layers. The first two layers have an input size of 128, and the third layer has an input size of 64. It transforms the features to achieve multimodal fusion. In the graph convolutional layers, two graph convolutional network layers are used with output sizes of 64 and 7, respectively, to extract graph-based features. The final modality classifier consists of a fully connected layer with an output size of 2, used to distinguish between different modalities.

[0132] In the model's hyperparameter settings, the batch size for both training and testing is set to 64, and the optimizer used is Adam. The learning rate is 0.0001, and the weight decay is 1×10⁻⁶. -4 The step size is 50. The training epochs are 100. Additionally, hyperparameters include λclass = 1.0 and λmodality = 0.4, used to balance the loss function for classification and modality tasks. The feature extraction window size is set to 1024, with 5 nearest neighbors, and each file contains 400 samples.

[0133] To evaluate the impact of each component in the MTMAGNet model proposed in this invention on the overall performance, ablation experiments were conducted by simplifying key modules, and the change in test accuracy with training iterations was observed. The setup methods for each ablation module are as follows: (a) To verify the effectiveness of the multimodal information fusion method, the fault diagnosis performance of single-modal data using only vibration signals and the GCN model combination was first evaluated, denoted as Only Vibration with GCN (OV-GCN); (b) Using only acoustic signals and the GCN model combination, denoted as Only Acoustic with GCN (OA-GCN); (c) Fusing acoustic and vibration signals but removing the modal attention mechanism while retaining the multi-task learning module, denoted as MTL-GCN; (d) Fusing acoustic and vibration signals but removing the multi-task learning module while retaining the modal attention mechanism, denoted as MMA-GCN; (e) Fusing acoustic and vibration signals but removing both the modal attention mechanism and the multi-task learning module, retaining only the GCN model, denoted as VA-GCN. The comparison results of different ablation methods are as follows: Figure 4 As shown. The significance of algorithmic ablation lies in its ability to quantitatively verify the necessity of a component in a model, evaluate its contribution, and gain a deeper understanding of the model's internal working mechanism by systematically removing or replacing that component. This, in turn, supports the effectiveness of core innovations, drives model simplification, and guides future research.

[0134] like Figure 5 As shown, to intuitively evaluate the performance of the proposed method, the confusion matrices of fault diagnosis results for different models (AGCN, Dual-GCN, GAT, CNN, LSTM) were investigated. In the confusion matrix, the vertical and horizontal axes represent the true health status and predicted health status of the detected samples, respectively. Figure 5 As can be seen, the MTMAGNet model achieves near-perfect classification across all fault categories, with no significant misclassifications, highlighting its robustness and accuracy in fault diagnosis. AGCN and Dual-GCN perform relatively well, but occasionally exhibit misclassifications, particularly in categories 3 and 5. GAT demonstrates competitive performance, but shows slightly lower accuracy in category 5. Traditional deep learning models CNN and LSTM exhibit the highest misclassification rates, especially in categories 2, 3, and 5, indicating their limitations in handling complex fault patterns. Overall, the MTMAGNet model outperforms other models in classification accuracy, demonstrating its effectiveness and reliability in intelligent fault diagnosis.

[0135] This invention proposes an intelligent bearing fault diagnosis method that fuses acoustic and vibration signals. Addressing the challenge of accurately detecting rolling bearing faults under complex operating conditions, the method effectively utilizes the complementary advantages of acoustic and vibration signals through the MTMAGNet fusion algorithm. Through a multimodal attention mechanism, the model dynamically adjusts the importance of different modes, thereby enhancing feature fusion and capturing fault characteristics. A multi-task learning framework further improves the model's generalization ability and prevents overfitting, enabling it to achieve near-100% classification accuracy under diverse fault conditions, surpassing existing state-of-the-art methods. Furthermore, the algorithm exhibits excellent noise robustness, maintaining high diagnostic accuracy even in highly challenging environments. These results validate the effectiveness and reliability of the MTMAGNet model in rolling bearing fault diagnosis, providing a promising solution for intelligent diagnosis in industrial applications.

[0136] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A bearing intelligent fault diagnosis method integrating acoustic and vibration signals, characterized in that, Rapid bearing fault diagnosis is achieved based on the analysis of fused acoustic and vibration signals, including the acquisition of acoustic and vibration signals, extraction and fusion, enhanced feature processing, model training, and generation of fault diagnosis results. Specifically, the steps include: 1) Acquire acoustic and vibration signals: acquire vibration signals generated during the operation of the rolling bearing through a vibration sensor, and acquire sound signals generated during the operation of the rolling bearing through a microphone array sensor; 2) Extraction and fusion: CNN is used to extract deep features from the collected sound and vibration signals. Then, MMA is applied to fuse these features. Attention weights are calculated to highlight the most relevant features in each modality, thereby obtaining the fused feature representation. 3) Enhanced feature processing: The fused feature vectors are used to construct a graph representation of the data, where nodes represent samples and edges are formed based on their similarity. The graph structure is created using the KNN algorithm. Then, the fused features and graph are passed to GCN. GCN captures the local and global relationships between samples, thereby generating an enhanced feature representation. 4) Model training: Input the GCN output into an MTL for fault classification. The MTL includes a main task and auxiliary tasks. Define a total loss function and train the model by minimizing this loss. 5) Generate fault diagnosis results, and use confusion matrix and classification scatter plot to intuitively evaluate the prediction effect of the model, and obtain accurate fault diagnosis results based on fused acoustic and vibration signals.

2. The bearing intelligent fault diagnosis method integrating acoustic and vibration signals according to claim 1, characterized in that, In the acquisition of acoustic and vibration signals in step 1), the sensor is arranged as follows: a vibration sensor is installed on the bearing seat of the bearing under test, and an acoustic array sensor consisting of 16 microphones is arranged on a plane 200 mm away from the end of the bearing under test, with the central axis of the array aligned with the axis of the bearing.

3. The bearing intelligent fault diagnosis method integrating acoustic and vibration signals according to claim 1, characterized in that, Step 2) involves extraction and fusion, including modality attention scoring, defining attention weights, and fusing feature vectors. The specific operations are as follows: 11) Extraction: Sound and vibration signals are processed through multiple one-dimensional convolutions to extract local patterns. Then, pooling layers are used for downsampling, and finally, a fully connected layer projects the final output onto a low-dimensional feature space. The extraction process can be described mathematically as follows: In the formula, This represents the sound signal value at the nth synchronization time point. Representing the vibration signal value at the same time point, Conv(•) represents the convolution operation, σ(•) is the activation function, Pool(•) represents the pooling operation, and the output is a d-dimensional feature vector; 12) Modal attention scores: The modal attention mechanism calculates the attention scores sa and sv for acoustic and vibration modes respectively, reflecting the relative importance of each mode. The specific formula is as follows: in, This represents the feature vector extracted from the sound signal. This represents the feature vector extracted from the vibration signal. It is a learnable weight matrix. For bias terms, This is a learnable attention vector; 13) Define attention weights. After obtaining the attention scores for the two modes, apply softmax normalization to calculate the attention weights, which represent the relative importance of each mode during the fusion process. The attention weights for the acoustic and vibration modes are defined as follows: In the formula, Sa and Sv are the attention scores for the acoustic mode and vibration mode, respectively; 14) Fusing Feature Vectors: After calculating the attention weights for the acoustic and vibration modes, the features of the two modes are weighted and summed to obtain the final fused feature vector. in, It is a fusion of feature vectors. and The attention weight for each modality is applied to the acoustic and vibration feature vectors respectively. The fused feature vector hfused combines information from both modalities, reflecting the different importance of each modal feature and providing a representation for subsequent classification tasks.

4. The bearing intelligent fault diagnosis method integrating acoustic and vibration signals according to claim 1, characterized in that, Step 3) Enhanced feature processing includes constructing a feature matrix, assigning neighbor weights, normalizing the adjacency matrix, and predicting node categories. The specific steps are as follows: 21) Constructing the feature matrix: A K-Nearest Neighbor (KNN) graph was built using fused features. Nodes were connected based on the similarity between samples. The feature matrix H, composed of the fused feature vectors hfused, is represented as follows: Where N represents the total number of samples and d represents the feature dimension of each sample; in order to construct the KNN graph, Euclidean distance is used to measure the similarity between samples, thereby determining the connection between nodes. For each node, K nearest neighbors are selected, and an adjacency matrix is ​​constructed based on the similarity between samples. Each edge in the adjacency matrix represents the similarity between connected samples. 22) Assign neighbor weights using a Graph Attention Network (GAT) with an attention mechanism. This network can assign different neighbor weights to each node. For node i, its aggregation features... Represented as: in, Let deg(i) represent the set of neighboring nodes of node i, and deg(j) represent the degrees of nodes i and j, respectively; W is the weight matrix shared by the entire layer. 23) Normalized adjacency matrix. The normalized adjacency matrix is ​​defined as follows: In the formula, A is the adjacency matrix of the graph, D is the degree matrix, and ; 24) Predicting Node Classes: After aggregating information from neighboring nodes, the output layer of GCN uses a classifier to predict the class of each node. The classification expression is as follows: In the formula, It is the weight matrix of the classifier. is the bias term of the classifier, and C is the number of classes.

5. The bearing intelligent fault diagnosis method integrating acoustic and vibration signals according to claim 1, characterized in that, Step 4) involves model training, which includes modal task classification and determining the total task loss. The specific steps are as follows: 31) Modal task classification: In a multi-task learning framework, the class label yclass of each sample is predicted first to determine its class. The fused features hfused are used to identify whether the features come from acoustic or vibration modes. The modal classifier used to identify the source modes of the fused features is calculated as follows: in, It is the weight matrix of the modal classifier, which projects the fused features onto a two-dimensional probability space, representing acoustic and vibration modes; It is the bias term of the modality classifier. It is a fused feature vector that integrates acoustic and vibrational characteristics; The actual label ymodality for each modality is defined as follows: Regarding acoustic characteristics: Regarding vibration characteristics: Through modality classification tasks, the model can better distinguish features from different modalities; 32) Determine the total task loss. In the multi-task learning process, the model's loss function consists of the main classification task loss and the auxiliary modality classification task loss. The final loss is the weighted sum of the main task loss and the auxiliary task loss. The loss function for the main classification task is cross-entropy loss, which measures the difference between the predicted class label and the true class label, and is defined as follows: In the formula, N is the number of samples, and C is the number of categories. This represents the true label of sample i for category j. It is the predicted probability of sample i belonging to category j from the model; The modality classification task is used to balance the model's accuracy in modality classification, and its loss function is defined as follows: In the formula, Let be the modal label for sample i. For acoustic features, the label is [1,0]; for vibration features, the label is [0,1]. It is the predicted probability of the model; The total loss is a weighted sum of the classification loss and the modality classification loss, and is expressed as: In the formula, λclassification (classification) and λmodality (modality) are weight parameters.

Images