A mechanical equipment fault diagnosis method and system based on sound signal understanding

Abstract

The application provides a mechanical equipment fault diagnosis method and system based on sound signal understanding, and belongs to the technical field of fault diagnosis. The method comprises the following steps: extracting first time-frequency features of a sound signal to be diagnosed through short-time Fourier transform, and extracting second time-frequency features of the sound signal to be diagnosed through a Mel spectrum; respectively performing standardization processing on the first time-frequency features and the second time-frequency features to obtain standardized first time-frequency features and standardized second time-frequency features; splicing the standardized first time-frequency features and the standardized second time-frequency features to obtain double-channel data to be diagnosed; and inputting the double-channel data to be diagnosed into a pre-trained neural network model to obtain a fault diagnosis result. The method significantly improves the precision, robustness and calculation efficiency of sound signal mechanical fault diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis technology, specifically relating to a method and system for diagnosing mechanical equipment faults based on sound signal understanding. Background Technology

[0002] Mechanical fault diagnosis technology. This type of technology is mainly applied to the monitoring of the operating status of rotating machinery (such as bearings, rollers, motors, etc.) in the industrial field. By collecting the sound signals generated during the operation of the machinery and combining them with signal processing and feature analysis methods, fault detection of equipment can be achieved under non-contact conditions, thereby reducing maintenance costs and improving operational safety and reliability.

[0003] In existing technologies, Liu Yi et al. disclosed a lightweight rotating machinery fault diagnosis method based on sound signals (Multi-branch feature fusion residual attention Pyramid Network, MPNet) in the paper "Measurement" (2025, 239: 115498). This method collects the operating sound of the equipment using a microphone array and converts the sound signal into a time-frequency feature map using short-time Fourier transform. The model design incorporates a multi-branch feature fusion module to extract local detail features and global correlation features in parallel. Simultaneously, a residual attention pyramid module is used to perform hierarchical weighted processing of the features, thereby achieving adaptive extraction of multi-scale fault features. Finally, a classifier is used to determine the fault type, realizing fault identification during the operation of rotating machinery. This technical solution achieved high recognition accuracy in experimental environments. However, due to the fixed size of the short-time Fourier transform window, its ability to capture transient changes in the sound signal is limited, making it difficult to fully represent dynamic features. Furthermore, in environments with complex noise or large changes in operating conditions, background noise easily interferes with fault feature extraction, thus affecting the stability of the recognition results.

[0004] In another study, Huang et al. proposed an abnormal sound detection method based on One-Stage Supervised Contrastive Learning (OS-SCL) in the paper *ICASSP* (2025: 1–5). This method mixes audio using MixUp and extracts features using MobileFaceNet. It introduces a feature perturbation mapping head in the embedding space to enhance discriminative ability, while combining time-frequency feature representation to improve information expression. During training, it uses a combination of noise-supervised contrastive loss and cross-entropy loss for joint optimization, thereby achieving effective detection of abnormal sounds from equipment. Although this approach improves the model's discriminative ability through feature perturbation and contrastive learning, it relies on large-scale datasets to ensure effectiveness. Its generalization ability is limited in scenarios with small samples or significant differences in operating conditions, and it has high computational resource requirements, making it difficult to deploy directly on resource-constrained industrial equipment.

[0005] In addition, other related studies have proposed a variety of innovative methods for fault diagnosis of industrial equipment. For example, the paper "Sensors" (2022, 22(9): 3275) proposed a pipeline robot fault diagnosis system (Fine-Tuned ResNet50 for Fault Diagnosis, hereinafter referred to as ResNet50) that combines sound signal recognition and deep transfer learning; the paper "Nonlinear Dynamics" (2025, 113: 22823–22852) proposed a fault diagnosis framework based on an improved multimodal data fusion method and the SWR2 Net network; and the paper "Information Fusion" (2025, 121: 103128) proposed a fault detection method based on a multilevel feature encoder (MLFE), which effectively solved the sample imbalance problem by utilizing transfer learning. These studies have explored the technical path of fault diagnosis based on sound signals from different perspectives and have made some progress.

[0006] However, a review of the aforementioned studies reveals that while existing methods have improved diagnostic accuracy to some extent, they generally suffer from insufficient ability to capture transient changes, susceptibility to noise interference, and strong dependence on data scale and computational resources. These issues can affect diagnostic accuracy and system reliability in complex industrial environments. Therefore, there is an urgent need to develop more efficient, robust, and adaptable fault diagnosis methods to better address application requirements under varying operating conditions and limited data. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this application proposes a mechanical equipment fault diagnosis method and system based on sound signal understanding. The aim is to solve the problems of weak anti-interference capability, insufficient transient feature extraction, and strong dependence on large-scale data and high computational resources in existing sound signal fault diagnosis methods. This application proposes a lightweight, high-precision, and robust mechanical fault diagnosis method. By optimizing the feature extraction structure and training strategy, the model maintains excellent fault identification performance under resource-constrained conditions, enabling real-time monitoring and rapid diagnosis of equipment operating status, thereby improving the safety and maintenance efficiency of industrial systems.

[0008] In a first aspect, the present invention provides a method for diagnosing mechanical equipment faults based on sound signal understanding, comprising:

[0009] Collect the sound signals to be diagnosed during the operation of mechanical equipment;

[0010] The first time-frequency feature is extracted from the sound signal to be diagnosed by short-time Fourier transform, and the second time-frequency feature is extracted from the sound signal to be diagnosed by Mel spectrum.

[0011] The first time-frequency feature and the second time-frequency feature are standardized respectively to obtain the standardized first time-frequency feature and the standardized second time-frequency feature.

[0012] The standardized first time-frequency feature and the standardized second time-frequency feature are spliced ​​together to obtain the dual-channel data to be diagnosed.

[0013] The dual-channel data to be diagnosed is input into a pre-trained neural network model to obtain fault diagnosis results. The training process of the pre-trained neural network model includes: extracting local texture features from the dual-channel data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain a first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain a second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain a feature tensor; inputting the feature tensor into a classifier to obtain a classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

[0014] The first time-frequency feature is extracted from the sound signal to be diagnosed using short-time Fourier transform, and the calculation formula is as follows:

[0015] ;

[0016] in, The sound signal to be diagnosed. For window functions, For a period of time, For linear frequency, The first time-frequency feature, The integral variable is the sound signal to be diagnosed.

[0017] The second time-frequency feature is extracted from the sound signal to be diagnosed using Mel-spectrum analysis, and the calculation formula is as follows:

[0018] ;

[0019] in, For linear frequency, This is the Mel frequency, i.e., the second time-frequency characteristic.

[0020] The step of inputting local texture features into a first depthwise separable convolution to obtain first mid-layer features includes:

[0021] A depthwise convolution operation is performed on each channel of the local texture features to extract the local spatial features of each channel, and a pointwise convolution operation is performed on the local spatial features of each channel to obtain the fused features of each channel.

[0022] Batch normalization of the fused features;

[0023] The normalized features are input into the LeakyReLU activation function for nonlinear mapping;

[0024] The features after nonlinear mapping are downsampled;

[0025] The downsampled features are processed using a discarding method to obtain the first middle-layer features.

[0026] The step of inputting the first mid-level features into the first residual attention module to obtain the first high-level feature representation includes:

[0027] The first middle layer feature is used as the main branch, and the identity mapping branch is copied to obtain the main branch. Two convolution operations are performed on the main branch to obtain the feature map.

[0028] Channel attention is calculated on the feature map to obtain a weighted feature map;

[0029] The weighted feature map is added element-wise to the identity mapping branch to obtain the summed feature map;

[0030] The summed feature maps are then passed through an activation function to obtain the first high-level feature representation.

[0031] The channel attention calculation of the feature map is performed to obtain a weighted feature map. The calculation process is as follows:

[0032] The channel attention mechanism extracts the statistical features of each channel of the feature map through global average pooling and global max pooling, as calculated below:

[0033] ;

[0034] ;

[0035] in, This is the statistical result of global average pooling. This is the statistical result of global max pooling. Let H be the element in the i-th row, j-th column, and c-th channel of the feature map, where H is the height of the feature map and W is the width of the feature map.

[0036] The attention weights for global average pooling and global max pooling are calculated using fully connected layers, as follows:

[0037] ;

[0038] ;

[0039] in, The attention weights are global average pooling. The attention weights are for global max pooling. It is the ReLU activation function. It is the Sigmoid activation function. The weight matrix of the fully connected layer with shared parameters in the first layer. This is the weight matrix of the fully connected layer with shared parameters in the second layer.

[0040] The attention weights of global average pooling and global max pooling are added together to obtain the final channel attention weights;

[0041] The feature map is weighted using the final channel attention weights to obtain a weighted feature map.

[0042] The step of inputting the second high-level feature representation into the multi-scale feature extraction layer to obtain the feature tensor includes:

[0043] The second high-level feature representation is convolved to obtain convolutional local features;

[0044] Batch normalization of local features from convolution;

[0045] Activate the batch-normalized features;

[0046] The activated features are subjected to average pooling to obtain compressed features.

[0047] The compressed features are processed using a discarding method to obtain the feature tensor.

[0048] Secondly, the present invention also provides a mechanical equipment fault diagnosis system based on sound signal understanding, comprising:

[0049] The data acquisition module is used to collect the sound signals to be diagnosed during the operation of mechanical equipment;

[0050] The feature extraction module is used to extract the first time-frequency features of the sound signal to be diagnosed through short-time Fourier transform and the second time-frequency features of the sound signal to be diagnosed through Mel spectrum.

[0051] The feature standardization module is used to standardize the first time-frequency feature and the second time-frequency feature respectively, so as to obtain the standardized first time-frequency feature and the standardized second time-frequency feature.

[0052] The feature splicing module is used to splice the standardized first time-frequency feature and the standardized second time-frequency feature to obtain the dual-channel data to be diagnosed.

[0053] The output module is used to input the dual-channel data to be diagnosed into a pre-trained neural network model to obtain the fault diagnosis result. The training process of the pre-trained neural network model includes: extracting local texture features from the dual-channel data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain the first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain the first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain the second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain the second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain the feature tensor; inputting the feature tensor into a classifier to obtain the classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

[0054] Thirdly, this application proposes an electronic device, comprising: one or more processors, and a memory communicatively connected to the one or more processors; the memory is used to store computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the aforementioned mechanical equipment fault diagnosis method based on sound signal understanding.

[0055] Fourthly, this application proposes a computer-readable storage medium storing computer-executable instructions thereon, which, when executed, cause the processor to perform the aforementioned mechanical equipment fault diagnosis method based on sound signal understanding.

[0056] Fifthly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implement the aforementioned method for diagnosing mechanical equipment faults based on sound signal understanding.

[0057] Beneficial effects:

[0058] This application proposes a method and system for mechanical equipment fault diagnosis based on sound signal understanding. The key innovative technology lies in significantly improving the accuracy, robustness, and computational efficiency of sound signal-based mechanical fault diagnosis by combining depthwise separable convolution, residual attention mechanism, and dual-spectrum input method. To verify the effectiveness of these techniques, experiments were conducted using data from the MIMII dataset under different signal-to-noise ratio conditions. Experimental results show that the method in this application has significant advantages in accuracy and computational efficiency compared to existing technologies. Attached Figure Description

[0059] Figure 1 A flowchart of a mechanical equipment fault diagnosis method based on sound signal understanding according to an embodiment of the present invention;

[0060] Figure 2 A schematic flowchart of a mechanical equipment fault diagnosis method based on sound signal understanding according to an embodiment of the present invention;

[0061] Figure 3 Neural network structure diagram of an embodiment of the present invention;

[0062] Figure 4 The accuracy of different models in this invention on the dataset;

[0063] Figure 5 Average prediction time of different models in embodiments of the present invention;

[0064] Figure 6 A block diagram of a mechanical equipment fault diagnosis system based on sound signal understanding according to an embodiment of the present invention. Detailed Implementation

[0065] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0066] Example 1:

[0067] This embodiment provides a method for diagnosing mechanical equipment faults based on sound signal understanding, such as... Figure 1 , Figure 2 As shown, it includes:

[0068] Step S1: Collect the sound signals to be diagnosed during the operation of the mechanical equipment;

[0069] In this embodiment, firstly, sound signals are acquired by collecting sound signals during the operation of the mechanical equipment using a microphone array, generating raw audio data as input for subsequent processing. In specific fault diagnosis, it is necessary to acquire the sound signals to be diagnosed during the operation of the mechanical equipment; however, in building a pre-trained neural network model, it is necessary to acquire historical sound signals from the mechanical equipment's operation. In this embodiment, after obtaining the historical raw audio data, a dataset is constructed, dividing the audio data into a training set and a test set, accounting for 80% and 20% respectively. A portion of the training set is further divided into a validation set to evaluate the model's performance and parameter adjustment effects during training. Data augmentation is performed on the audio data in the training set: Gaussian noise is added to simulate a variable noise environment and improve the model's robustness; simultaneously, time shifting, gain variation, random pruning, and zero padding are applied to the audio data to enrich the diversity of the training data.

[0070] Step S2: Extract the first time-frequency feature from the sound signal to be diagnosed using short-time Fourier transform, and extract the second time-frequency feature from the sound signal to be diagnosed using Mel spectrum;

[0071] In this embodiment, for the sound signal to be diagnosed or the historical sound signal after data enhancement, the time-frequency features of the audio data are extracted by Short Time Fourier Transform (STFT) and Mel-spectrogram, respectively.

[0072] The first time-frequency feature is extracted from the sound signal to be diagnosed using short-time Fourier transform, and the calculation formula is as follows:

[0073] ;

[0074] in, The sound signal to be diagnosed. For window functions, For a period of time, For linear frequency, The first time-frequency feature, This is the integral variable of the sound signal to be diagnosed. STFT transforms the signal from the time domain to the time-frequency domain, helping to capture instantaneous changes that occur during the operation of mechanical equipment.

[0075] The calculation process of the Mel spectrum includes first extracting the signal spectrum through STFT, then converting the spectrum to the Mel scale, and finally extracting the second time-frequency feature from the audio signal to be diagnosed using the Mel spectrum. The calculation formula is as follows:

[0076] ;

[0077] in, For linear frequency, This is the Mel frequency, i.e., the second time-frequency characteristic.

[0078] It can be understood that: by extracting the signal spectrum through STFT, and based on the frequency index k in the spectrum, using the sampling rate and the number of points in the FFT (Fast Fourier Transform), the linear frequency corresponding to each frequency index k is calculated, i.e., the linear frequency. The calculation formula is as follows:

[0079] ;

[0080] Where k is the frequency index in the spectrum. Sampling rate, This represents the number of points in the Fast Fourier Transform.

[0081] Step S3: Standardize the first time-frequency feature and the second time-frequency feature respectively to obtain the standardized first time-frequency feature and the standardized second time-frequency feature;

[0082] In this embodiment, the two obtained time-frequency features are standardized to ensure that the mean of the data is 0 and the variance is 1, so as to eliminate the dimensional differences between different features.

[0083] Step S4: The standardized first time-frequency feature and the standardized second time-frequency feature are spliced ​​together to obtain the dual-channel data to be diagnosed;

[0084] In this embodiment, after standardization, bilinear interpolation is performed on the two time-frequency features to ensure consistent dimensions. Bilinear interpolation, by weighting the average of four adjacent sampling points, can smooth the transition features, preserve structural continuity, and avoid information distortion. Finally, the two time-frequency features are concatenated to form the dual-channel data to be diagnosed, which serves as the input data for the pre-trained neural network model. The training data is also processed through steps S2 to S4 to obtain the final dual-channel training data, which is used to train the neural network model.

[0085] Step S5: Input the dual-channel data to be diagnosed into the pre-trained neural network model to obtain the fault diagnosis result. The pre-trained neural network model has the following structure: Figure 3As shown, the training process includes: extracting local texture features from the dual-channel training data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain the first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain the first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain the second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain the second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain a feature tensor; inputting the feature tensor into a classifier to obtain the classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

[0086] In this embodiment, the pre-trained neural network model consists of a feature extraction layer, a residual attention module, a multi-scale fusion layer, and a classifier.

[0087] The training process of the pre-trained neural network model includes:

[0088] Step S5.1: The preparation phase for training, including steps S5.1.1 to S5.1.7;

[0089] Step S5.1.1: Initialize the neural network model and set the model parameters.

[0090] Step S5.1.2: Set training parameters, such as the number of training epochs, initial learning rate, and early stopping patience value.

[0091] Step S5.1.3: Set the loss function for the neural network model. This embodiment introduces the Focal Loss function to focus on the contribution of minority class samples (i.e., faulty samples) to the loss function.

[0092] The formula for the focus loss function is as follows:

[0093] ;

[0094] in, For focus loss function, The confidence level when the model's prediction matches the true label. This is a balancing factor. It is used to adjust the weights of different classes of samples contributing to the overall loss, thereby preventing the minority class from being ignored during training. Meanwhile, This is called the focusing term, among which It is an adjustable hyperparameter.

[0095] Step S5.1.4: Set up the optimizer. This embodiment uses the AdamW optimizer to update the parameters of the neural network.

[0096] Step S5.1.5: Set up the Gradient Scaling tool to accelerate the training process of the neural network.

[0097] Step S5.1.6: Set the learning rate scheduler. This embodiment uses the ReduceLROnPlateau learning rate scheduler, which automatically reduces the learning rate if the validation set performance does not improve significantly over multiple epochs.

[0098] Step S5.1.7: Set up an early stopping mechanism. If the performance on the validation set does not improve within a certain number of training epochs, training is stopped to prevent the model from overfitting.

[0099] Step S5.2: Enter the training loop and begin training the neural network model;

[0100] Step S5.2.1: The neural network receives preprocessed dual-channel time-frequency feature input (STFT and Mel concatenation). First, low-level features are extracted through an initial convolutional block, and the number of channels is expanded to 32 layers. This module consists of 3×3 convolutional layers, batch normalization layers, LeakyReLU activation, and Dropout. It is used to capture local texture features of the sound signal, improve nonlinear expressive power, and suppress overfitting, providing a stable feature foundation for subsequent deep separable convolution and residual attention learning.

[0101] Step S5.2.2: Input the local texture features into the first depthwise separable convolution layer to obtain the first intermediate layer features, including:

[0102] Step S5.2.2.1: Perform a depthwise convolution operation on each channel of the local texture features to extract the local spatial features of each channel, and perform pointwise convolution operation on the local spatial features of each channel to obtain the fused features of each channel;

[0103] In this embodiment, it can be understood that the texture features consist of multiple channels, each of which is a two-dimensional matrix, and the depthwise convolution operation is performed on the two-dimensional matrix. In depthwise convolution, each convolution kernel processes one channel independently. Compared to ordinary convolution where each convolution kernel participates in the operation of all channels, depthwise convolution can better extract local features of each channel, while significantly reducing the computational complexity and number of parameters of the model.

[0104] Step S5.2.2.2: Perform batch normalization on the fused features;

[0105] Step S5.2.2.3: Input the normalized features into the LeakyReLU activation function for nonlinear mapping;

[0106] Step S5.2.2.4: Downsample the features after nonlinear mapping;

[0107] Step S5.2.2.5: Process the downsampled features using the discard method to obtain the first middle layer features.

[0108] In this embodiment, the features processed by the initial convolutional block are input to the first-layer depthwise separable convolutional module. This module first performs depthwise convolution operations on each channel, sliding the convolution kernel within a fixed channel to extract local spatial features; then, it performs pointwise convolution operations (1×1 convolution) to achieve linear combination and information fusion between channels, expanding the number of channels to 64. After two levels of convolution, the output features are sequentially processed by batch normalization to stabilize the distribution, the LeakyReLU activation function introduces a non-linear mapping, and then downsampling is performed through a 2×2 max pooling layer to reduce the feature size. Dropout is used to suppress overfitting, resulting in a mid-layer feature representation used as input to the residual attention module.

[0109] Step S5.2.3: The step of inputting the first mid-level features into the first residual attention module to obtain the first high-level feature representation includes:

[0110] Step S5.2.3.1: Take the first middle layer feature as the main branch and copy it to obtain the identity mapping branch. Perform two convolution operations on the main branch in sequence to obtain the feature map;

[0111] Step S5.2.3.2: Perform channel attention calculation on the feature map to obtain a weighted feature map;

[0112] Step S5.2.3.3: Add the weighted feature map to the identity mapping branch element by element to obtain the added feature map;

[0113] Step S5.2.3.4: Pass the summed feature maps through an activation function to obtain the first high-level feature representation.

[0114] In this embodiment, the features processed by the first depthwise separable convolution are input into the first set of residual attention modules. First, the input features are copied into an identity mapping branch (denoted as input x), and two convolution operations are performed on the main branch in sequence. Each convolution includes a 3×3 convolutional layer and a batch normalization layer to obtain a feature map F(x), which is used to extract higher-level feature representations, namely the first high-level feature representation.

[0115] Step S5.2.3.2 involves calculating channel attention on the feature map to obtain a weighted feature map. The calculation process is as follows:

[0116] The channel attention mechanism extracts the statistical features of each channel of the feature map through global average pooling and global max pooling, as calculated below:

[0117] ;

[0118] ;

[0119] in, This is the statistical result of global average pooling. This is the statistical result of global max pooling. Let H be the element in the i-th row, j-th column, and c-th channel of the feature map, where H is the height of the feature map and W is the width of the feature map.

[0120] In this embodiment, the attention weights for each pooling result are calculated using a fully connected layer. and The final channel attention coefficients are generated using the Sigmoid activation function.

[0121] The attention weights for global average pooling and global max pooling are calculated using fully connected layers, as follows:

[0122] ;

[0123] ;

[0124] in, The attention weights are global average pooling. The attention weights are for global max pooling. It is the ReLU activation function. It is the Sigmoid activation function. The weight matrix of the fully connected layer with shared parameters in the first layer. This is the weight matrix of the fully connected layer with shared parameters in the second layer.

[0125] The attention weights of global average pooling and global max pooling are added together to obtain the final channel attention weights;

[0126] ;

[0127] in, This represents the final channel attention weights.

[0128] The feature map is weighted using the final channel attention weights to obtain a weighted feature map;

[0129] ;

[0130] in, For weighted feature maps, This is a feature map.

[0131] In step S5.2.3.3: the weighted feature map is added element-wise to the identity mapping branch to obtain the added feature map. The process is as follows:

[0132] In this embodiment, residual connections are applied to transform the original input feature map. (i.e., the identity mapping branch) and the weighted feature map Adding them together enhances the ability to learn important features while retaining input information:

[0133] ;

[0134] in, This is the identity mapping branch. This is the feature map after addition.

[0135] In step S5.2.3.4: the summed feature maps are then passed through an activation function to obtain the first high-level feature representation. The process is as follows:

[0136] Finally, the feature map after residual connection The final output feature map is obtained by performing a nonlinear transformation using an activation function (such as LeakyReLU).

[0137] ;

[0138] in, This represents the first high-level feature expression. This is the activation function.

[0139] Step S5.2.4: Input the first high-level feature representation into the second depthwise separable convolution to obtain the second middle-level feature;

[0140] In this embodiment, the features output by the first set of residual attention modules are input to the second layer of depthwise separable convolutional modules. This module expands the number of input channels from 64 to 128. First, depthwise convolution is performed independently on each channel, followed by pointwise convolution to complete channel fusion. The output features are then processed sequentially through batch normalization, LeakyReLU activation, 2×2 max pooling, and Dropout to obtain a feature map for subsequent residual attention calculation. The specific method is similar to obtaining the first intermediate layer features, and will not be described in detail in this embodiment.

[0141] Step S5.2.5: Input the second middle-layer features into the second residual attention module to obtain the second high-layer feature representation;

[0142] In this embodiment, the second mid-layer features are input to the second set of residual attention modules. Similar to the first set of residual attention modules, the input is first copied to the identity mapping branch, and two 3×3 convolution and batch normalization operations are performed sequentially on the main branch to obtain the feature map F(x). Then, channel attention is calculated on F(x) to obtain a weighted feature map. Finally, the feature map is added element-wise with the identity mapping branch and output through the LeakyReLU activation function as the input to the subsequent multi-scale feature extraction layer. The specific method is similar to that used to obtain the second high-level feature representation, and will not be described in detail in this embodiment.

[0143] Step S5.2.6: Input the second high-level feature representation into the multi-scale feature extraction layer to obtain the feature tensor, including:

[0144] Step S5.2.6.1: Perform convolution operation on the second high-level feature representation to obtain convolutional local features;

[0145] Step S5.2.6.2: Perform batch normalization on the local features of the convolution;

[0146] Step S5.2.6.3: Perform activation operation on the batch normalized features;

[0147] Step S5.2.6.4: Perform average pooling on the activated features to obtain compressed features;

[0148] Step S5.2.6.5: Perform operations on the compressed features using the discarding method to obtain the feature tensor.

[0149] In this embodiment, after passing through two layers of depthwise separable convolution and residual attention modules, the resulting second high-level feature representation is input to the multi-scale feature extraction layer. This layer first performs 3×3 convolution operations to further integrate local features, followed by batch normalization and LeakyReLU activation; then, it performs spatial compression of the feature map through adaptive average pooling (4×4), and finally applies a Dropout operation, outputting a feature tensor of size 256×4×4, which serves as the input to the classifier.

[0150] Step S5.2.7: Input the feature tensor into the classifier to obtain the classification result. The process is as follows:

[0151] The feature tensor is flattened into a one-dimensional vector and then input into the classifier. The classifier consists of three fully connected layers, with LeakyReLU activation and Dropout operations applied between each layer. The first fully connected layer maps the input dimension from 256×4×4 to 512, the second layer maps it from 512 to 256, and the third layer outputs the predicted number of fault categories, which is used to complete the model's classification calculation.

[0152] Step S5.2.8: Calculate the loss value based on the classification result and the true label. Then, perform backpropagation based on the loss value to update the model parameters in the neural network model. The process is as follows:

[0153] After forward propagation, the loss value is calculated based on the classification results output by the neural network model and the true labels. The output predictions and labels are then input into the Focal Loss function to obtain the loss for the current batch, which is used for backpropagation and parameter updates.

[0154] Step S5.2.9: Perform backpropagation based on the calculated loss value. After amplifying the gradient using a gradient scaling tool, calculate the backpropagation gradient of each parameter; then descale and perform gradient clipping to prevent excessively large gradients from causing training instability.

[0155] Step S5.2.10: After each training batch is completed, evaluate the model using the validation set, and calculate the validation accuracy and loss value. If the current validation accuracy is higher than the historical best record, update the best accuracy and save the corresponding model parameters;

[0156] Step S5.2.11: Call the learning rate scheduler to monitor the performance of the validation set. When the validation accuracy does not improve within a set number of consecutive rounds, automatically adjust the learner's learning rate and reduce it by a preset ratio to promote the convergence of the model in subsequent training.

[0157] Step S5.2.12: Invoke the early stopping mechanism. When the accuracy has not improved after exceeding the set number of rounds, the model stops training and jumps to step S5.2.14 for final fine-tuning.

[0158] Step S5.2.13: After completing the current training round, determine whether the number of training cycles executed has reached the preset total number of rounds; if not, return to step S5.2 to continue subsequent training.

[0159] Step S5.2.14: After training is completed, load the best model parameters saved during training, and use all training data (including training set and validation set) to perform several rounds of fine-tuning training to make full use of all sample information and further optimize model performance.

[0160] Step S5.2.14: After the fine-tuning process is completed, save the final optimized model parameters (i.e., the pre-trained neural network model) and generate a complete model file. This model serves as the final version and is used for inference and fault category prediction of unknown sound signals.

[0161] Once the model training is complete, the saved pre-trained neural network model can be loaded to predict new mechanical equipment sound signals (i.e., sound signals to be diagnosed).

[0162] After loading the pre-trained neural network model, the sound signal to be diagnosed is used as input, and preprocessing steps and feature extraction (including steps S2 to S4) are performed in sequence to generate dual-channel data to be diagnosed that meet the model input requirements.

[0163] The dual-channel data to be diagnosed is input into a pre-trained neural network model, which then outputs the corresponding fault category prediction result through forward inference calculation.

[0164] To verify the effectiveness of the method in this embodiment, the experimental procedure is as follows:

[0165] The MIMII dataset is a publicly available dataset for mechanical fault diagnosis, covering various types of industrial equipment faults (such as fans, water pumps, etc.). The dataset includes audio signals under different signal-to-noise ratio (SNR) conditions. We selected data with a SNR of 6 dB (i.e., clean data) from this dataset for training. The training data was internally divided into training and validation sets for real-time evaluation of the model's training effectiveness and performance. Testing was conducted under SNR conditions of 6 dB, 0 dB, and -6 dB, and the average accuracy and average prediction time per sample for each model were calculated. These settings help to comprehensively evaluate the model's performance under different noise backgrounds, especially its robustness in low SNR environments.

[0166] In terms of diagnostic accuracy, the DSRANet model (Depthwise Separable Residual Attention Network) in this embodiment outperforms the comparison model under various signal-to-noise ratio conditions. For example... Figure 4As shown, MPNet (Multi-branch feature fusion residual attention Pyramid Network), OS-SCL (One-Stage Supervised Contrastive Learning), ResNet50 (Fine-Tuned ResNet50 for Fault Diagnosis), SWR2 Net (Shifted Window attention improved multi-scale residual network), and MLFE (Multilevel Feature Encoder) were used. Under high signal-to-noise ratio (SNR) conditions (6dB, 0dB), DSRANT's accuracy was slightly higher than other models; while under low SNR conditions (−6dB), its accuracy reached 92.30%, an improvement of about 10% over the second-best model. The results indicate that DSRANT can maintain a high level of fault identification capability even in environments with significant noise interference, demonstrating excellent robustness and stability.

[0167] In terms of computational efficiency, the DSRANet model proposed in this embodiment exhibits superior inference performance. For example... Figure 5 As shown, DSRANT's average prediction time is 0.011 s, which is about 21% faster than SWR2Net (0.014 s), the fastest model in the comparison, demonstrating higher computational efficiency. Compared to other models (such as MPNet and MLFE), DSRANT also shows a significant time advantage. The above results indicate that the model proposed in this embodiment significantly improves computational efficiency while maintaining high diagnostic accuracy, providing a more feasible solution for real-time fault diagnosis in practical industrial scenarios.

[0168] Furthermore, we performed detailed calculations on the precision, recall, and F1 score of each model on this dataset under different signal-to-noise ratio conditions, and obtained the average values ​​of these metrics. Precision measures the proportion of samples predicted as positive that are actually positive; recall measures the proportion of samples that are actually positive that are correctly predicted; and the F1 score is the harmonic mean of precision and recall, which comprehensively considers the model's accuracy and sensitivity. These comprehensive evaluation metrics allow for a more complete analysis and comparison of the model's performance and robustness in noisy environments. The experimental results are shown in Table 1. The data in the table clearly shows that the model of this invention performs excellently on all evaluation metrics, significantly outperforming other comparative models, which fully demonstrates its superior performance in complex noisy environments. These results indicate that the model of this invention not only possesses excellent diagnostic accuracy but also exhibits strong robustness, effectively coping with diverse noise interference in industrial environments, and has broad application prospects.

[0169] Table 1: Experimental Results

[0170] ;

[0171] This embodiment proposes a mechanical equipment fault diagnosis method based on sound signal understanding. The innovation of this embodiment lies in significantly improving the accuracy and robustness of sound signal-based mechanical fault diagnosis by combining depthwise separable convolution, residual channel attention mechanism, and dual-spectrum input technology. These techniques effectively overcome the shortcomings of existing methods in complex noise environments, improve the accuracy of fault diagnosis, and reduce the computational burden, making it more suitable for industrial field applications.

[0172] In the data preprocessing stage, this embodiment employs a dual-spectrum fusion strategy, combining Short-Time Fourier Transform (STFT) and Mel-spectrogram to construct dual-channel input features. STFT provides precise information on signal variations in both time and frequency dimensions, capturing transient characteristics during mechanical operation; Mel-spectrogram, based on human auditory perception, provides higher resolution in the low-frequency region, more sensitively reflecting energy differences in equipment fault sounds. By complementary fusion of these two spectral features, the model can simultaneously perceive key information across different frequency bands, achieving a comprehensive representation of time-frequency features, thereby significantly improving fault feature extraction capabilities and diagnostic accuracy.

[0173] For feature extraction, this embodiment employs depthwise separable convolution. This technique significantly reduces computational complexity by decomposing traditional convolution into depthwise convolution and pointwise convolution, while retaining the ability to effectively extract features from the signal. Depthwise separable convolution enables the model to extract discriminative features from complex audio signals, making it particularly suitable for resource-constrained industrial environments.

[0174] To further enhance the model's ability to model the importance of feature channels, this embodiment introduces a channel attention mechanism before the residual connection. Channel attention adaptively redistributes features through global pooling and weighted operations, strengthening key channels. Subsequently, the weighted features are added to the identity mapping branch using residuals, achieving effective information fusion and stable gradient propagation, thereby improving the model's response to fault features while maintaining feature integrity.

[0175] During training, this embodiment employs a focal loss function to address the class imbalance problem, enabling the model to better focus on hard-to-classify samples. To further optimize the training process, adaptive learning rate scheduling (ReduceLROnPlateau) and gradient scaling (GradScaler) are also combined. These techniques accelerate the training process and ensure that the gradient overflow problem is effectively resolved, thereby improving training stability, especially when using low-precision computation.

[0176] In summary, this embodiment proposes an efficient, stable, and computationally-light method for diagnosing mechanical faults using sound signals. It can operate stably in complex noisy environments and provides real-time monitoring and fault warnings for industrial equipment, significantly improving equipment safety and reliability.

[0177] Example 2:

[0178] This embodiment also provides a mechanical equipment fault diagnosis system based on sound signal understanding, such as... Figure 6 As shown, it includes:

[0179] The system comprises a data acquisition module, a feature extraction module, a feature standardization module, a feature splicing module, and a result output module, which are connected sequentially.

[0180] The data acquisition module is used to collect the sound signals to be diagnosed during the operation of mechanical equipment;

[0181] The feature extraction module is used to extract the first time-frequency features of the sound signal to be diagnosed through short-time Fourier transform and the second time-frequency features of the sound signal to be diagnosed through Mel spectrum.

[0182] The feature standardization module is used to standardize the first time-frequency feature and the second time-frequency feature respectively, so as to obtain the standardized first time-frequency feature and the standardized second time-frequency feature.

[0183] The feature splicing module is used to splice the standardized first time-frequency feature and the standardized second time-frequency feature to obtain the dual-channel data to be diagnosed.

[0184] The output module is used to input the dual-channel data to be diagnosed into a pre-trained neural network model to obtain the fault diagnosis result. The training process of the pre-trained neural network model includes: extracting local texture features from the dual-channel data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain the first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain the first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain the second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain the second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain the feature tensor; inputting the feature tensor into a classifier to obtain the classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

[0185] Example 3:

[0186] This embodiment proposes an electronic device, including: one or more processors, and a memory communicatively connected to the one or more processors; the memory is used to store instructions, which, when executed by the one or more processors, cause the one or more processors to perform the aforementioned mechanical equipment fault diagnosis method based on sound signal understanding.

[0187] The electronic device can be a terminal device such as a mobile phone, computer, or tablet computer, and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a mechanical equipment fault diagnosis method based on sound signal understanding as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0188] The processor is used to execute all or part of the steps in the mechanical equipment fault diagnosis method based on sound signal understanding as described in the above embodiments. The memory is used to store various types of data, including but not limited to instructions for any application or method in the electronic device, as well as application-related data.

[0189] The processor can be implemented using one or more of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, to execute the mechanical equipment fault diagnosis method based on sound signal understanding described in the above embodiments.

[0190] Example 4:

[0191] This embodiment proposes a computer-readable storage medium storing a computer program / instructions thereon. When the computer program / instructions are executed by a processor, they implement the steps of a mechanical equipment fault diagnosis method based on sound signal understanding as described in the foregoing embodiments. The computer program product can be stored in a computer-readable storage medium, which includes several instructions to cause a computer device (which may be a personal computer, server, network device, or mobile terminal, etc.) to perform all or part of the steps of the methods described in the various embodiments of this application.

[0192] The aforementioned computer-readable storage media include, but are not limited to: flash memory, hard disk drive (HDD), solid state drive (SSD), multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or memory stick), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc.

[0193] Example 5:

[0194] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned method for diagnosing mechanical equipment faults based on sound signal understanding.

[0195] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0196] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0197] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of equivalent technology of this disclosure, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A method for diagnosing mechanical equipment faults based on sound signal understanding, characterized in that, include: Collect the sound signals to be diagnosed during the operation of mechanical equipment; The first time-frequency feature is extracted from the sound signal to be diagnosed by short-time Fourier transform, and the second time-frequency feature is extracted from the sound signal to be diagnosed by Mel spectrum. The first time-frequency feature and the second time-frequency feature are standardized respectively to obtain the standardized first time-frequency feature and the standardized second time-frequency feature. The standardized first time-frequency feature and the standardized second time-frequency feature are spliced ​​together to obtain the dual-channel data to be diagnosed. The dual-channel data to be diagnosed is input into a pre-trained neural network model to obtain fault diagnosis results. The training process of the pre-trained neural network model includes: extracting local texture features from the dual-channel data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain a first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain a second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain a feature tensor; inputting the feature tensor into a classifier to obtain a classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

2. The mechanical equipment fault diagnosis method based on sound signal understanding according to claim 1, characterized in that, The first time-frequency feature is extracted from the sound signal to be diagnosed using short-time Fourier transform, and the calculation formula is as follows: ； in, The sound signal to be diagnosed. For window functions, For a period of time, For linear frequency, The first time-frequency feature, The integral variable is the sound signal to be diagnosed.

3. The mechanical equipment fault diagnosis method based on sound signal understanding according to claim 1, characterized in that, The second time-frequency feature is extracted from the sound signal to be diagnosed using Mel-spectrum analysis, and the calculation formula is as follows: ； in, For linear frequency, This is the Mel frequency, i.e., the second time-frequency characteristic.

4. The mechanical equipment fault diagnosis method based on sound signal understanding according to claim 1, characterized in that, The step of inputting local texture features into a first depthwise separable convolution to obtain first mid-layer features includes: A depthwise convolution operation is performed on each channel of the local texture features to extract the local spatial features of each channel, and a pointwise convolution operation is performed on the local spatial features of each channel to obtain the fused features of each channel. Batch normalization of the fused features; The normalized features are input into the LeakyReLU activation function for nonlinear mapping; The features after nonlinear mapping are downsampled; The downsampled features are processed using a discarding method to obtain the first middle-layer features.

5. The method for diagnosing mechanical equipment faults based on sound signal understanding according to claim 1, characterized in that, The step of inputting the first mid-level features into the first residual attention module to obtain the first high-level feature representation includes: The first middle layer feature is used as the main branch, and the identity mapping branch is copied to obtain the main branch. Two convolution operations are performed on the main branch to obtain the feature map. Channel attention is calculated on the feature map to obtain a weighted feature map; The weighted feature map is added element-wise to the identity mapping branch to obtain the summed feature map; The summed feature maps are then passed through an activation function to obtain the first high-level feature representation.

6. The mechanical equipment fault diagnosis method based on sound signal understanding according to claim 5, characterized in that, The channel attention calculation of the feature map is performed to obtain a weighted feature map. The calculation process is as follows: The channel attention mechanism extracts the statistical features of each channel of the feature map through global average pooling and global max pooling, as calculated below: ； ； in, This is the statistical result of global average pooling. This is the statistical result of global max pooling. Let H be the element in the i-th row, j-th column, and c-th channel of the feature map, where H is the height of the feature map and W is the width of the feature map. The attention weights for global average pooling and global max pooling are calculated using fully connected layers, as follows: ； ； in, The attention weights are global average pooling. The attention weights are for global max pooling. It is the ReLU activation function. It is the Sigmoid activation function. The weight matrix of the fully connected layer with shared parameters in the first layer. This is the weight matrix of the fully connected layer with shared parameters in the second layer; The attention weights of global average pooling and global max pooling are added together to obtain the final channel attention weights; The feature map is weighted using the final channel attention weights to obtain a weighted feature map.

7. The mechanical equipment fault diagnosis method based on sound signal understanding according to claim 1, characterized in that, The step of inputting the second high-level feature representation into the multi-scale feature extraction layer to obtain the feature tensor includes: The second high-level feature representation is convolved to obtain convolutional local features; Batch normalization of local features from convolution; Activate the batch-normalized features; The activated features are subjected to average pooling to obtain compressed features. The compressed features are processed using a discarding method to obtain the feature tensor.

8. A mechanical equipment fault diagnosis system based on sound signal understanding, implemented using the mechanical equipment fault diagnosis method based on sound signal understanding as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to collect the sound signals to be diagnosed during the operation of mechanical equipment; The feature extraction module is used to extract the first time-frequency features of the sound signal to be diagnosed through short-time Fourier transform and the second time-frequency features of the sound signal to be diagnosed through Mel spectrum. The feature standardization module is used to standardize the first time-frequency feature and the second time-frequency feature respectively, so as to obtain the standardized first time-frequency feature and the standardized second time-frequency feature. The feature splicing module is used to splice the standardized first time-frequency feature and the standardized second time-frequency feature to obtain the dual-channel data to be diagnosed. The output module is used to input the dual-channel data to be diagnosed into a pre-trained neural network model to obtain the fault diagnosis result. The training process of the pre-trained neural network model includes: extracting local texture features from the dual-channel data using convolution; inputting the local texture features into a first depthwise separable convolution layer to obtain the first mid-layer features; inputting the first mid-layer features into a first residual attention module to obtain the first high-level feature representation; inputting the first high-level feature representation into a second depthwise separable convolution layer to obtain the second mid-layer features; inputting the second mid-layer features into a second residual attention module to obtain the second high-level feature representation; inputting the second high-level feature representation into a multi-scale feature extraction layer to obtain the feature tensor; inputting the feature tensor into a classifier to obtain the classification result; calculating the loss value based on the classification result and the true label; and updating the model parameters in the neural network model through backpropagation based on the loss value to obtain the pre-trained neural network model.

9. An electronic device, characterized in that, include: One or more processors, and a memory communicatively connected to the one or more processors; The memory is used to store computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the mechanical equipment fault diagnosis method based on sound signal understanding as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, which, when executed, cause the processor to perform a mechanical equipment fault diagnosis method based on sound signal understanding as described in any one of claims 1 to 7.

Images