A motor abnormal sound detection method based on feature aggregation transformer combined with DenseNet

Abstract

The application discloses a motor abnormal sound detection method based on feature aggregation Transformer combined with DenseNet, which comprises feature extraction, pretreatment, position coding, Transformer global time sequence modeling, multi-branch DenseNet local feature enhancement, average pooling, multi-scale feature fusion and classification; the DenseNet module realizes the strengthened expression of local features on the output of each layer of the Transformer, makes up for the deficiency of the Transformer in fine-grained local acoustic feature extraction, unifies the global time sequence modeling and local fine-grained feature enhancement of the motor sound signal, realizes multi-scale feature fusion, and thus improves the accuracy of motor abnormal sound detection.

Description

Technical Field

[0001] This invention relates to an abnormal detection technology for motor equipment, and more particularly to a method for detecting abnormal motor sounds based on a combination of multi-layer feature aggregation Transformer and DenseNet. Background Technology

[0002] Electric motors are essential electromechanical devices in modern production and daily life, playing a vital role in the smooth operation of industrial processes and the safe operation of equipment. However, manufactured motors are prone to abnormal conditions, leading to malfunctions and safety hazards. Quickly detecting abnormal equipment can reduce the number of defective products and prevent further damage. The sound of a motor's operation can be used to determine whether it is normal or abnormal. For manual inspection, checking for abnormal sounds and other characteristics would consume a significant amount of human and material resources, while machine identification of abnormal equipment can reduce the losses in human resources and finances.

[0003] Common machine learning models used for motor sound classification include RNNs, CNNs, and Transformers. Compared to traditional neural networks, Transformers have significant advantages in processing sequential data (such as sound and text), mainly in the following aspects: First, the self-attention mechanism allows it to focus on all positions in the entire sequence, overcoming the disadvantages of RNNs and CNNs in long-range dependencies; second, it has strong parallel computing capabilities and faster training speed; third, it is efficient inference and has lower computational resource requirements. Based on these advantages, Transformers have been widely used in speech recognition, sound classification, and large-scale dialogue language models (such as ChatGPT and DeepSeek). Transformers have excellent global temporal modeling capabilities and can effectively learn long-range dependencies in sound sequences, but their ability to extract local fine-grained acoustic features is relatively limited. On the other hand, convolutional models such as DenseNet can effectively extract local time-frequency features, but their receptive field is limited, and their ability to model global temporal relationships in sound is insufficient, making it difficult to fully capture anomalous features across time scales.

[0004] Existing methods only utilize single-layer features for classification, lacking the joint utilization of semantic information at different levels. This results in insufficient robustness and generalization ability of the model for abnormal sounds under complex operating conditions. DenseNet, on the other hand, achieves feature reuse through a densely connected structure, which can enhance the expressive power of local features and improve model accuracy. Therefore, combining Transformer with DenseNet helps to achieve joint global and local modeling of abnormal motor sounds. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a method for detecting abnormal motor sounds based on a combination of multi-layer feature aggregation Transformer and DenseNet.

[0006] Technical solution: The present invention provides a method for detecting abnormal motor sounds by combining feature aggregation Transformer and DenseNet, comprising:

[0007] (1) Feature extraction: The time-domain signal of the motor sound is collected and processed by the MFCC feature extraction algorithm to obtain the spectrum feature matrix;

[0008] (2) Preprocessing: Input the spectral feature matrix into the preprocessing fully connected layer to perform feature dimension transformation;

[0009] (3) Position encoding: Input the preprocessed feature matrix into the sine position encoding module, and add position information to the feature matrix through a sine function;

[0010] (4) Transformer global temporal modeling: Input the feature matrix with added position encoding into each Transformer layer to obtain the feature matrix of each Transformer layer;

[0011] (5) Multi-branch DenseNet local feature enhancement: The feature matrix of each Transformer layer is input in parallel to the corresponding DenseNet module, and then the feature dimension is compressed through a DenseNet fully connected layer.

[0012] (6) Average pooling: The fully connected feature matrices output by each DenseNet fully connected layer are compressed in temporal dimension by average pooling layers.

[0013] (7) Multi-scale feature fusion: The global feature vectors after average pooling of each layer are stacked layer by layer to obtain a multi-scale feature matrix;

[0014] (8) Classification: Input the multi-scale feature matrix into the binary classification fully connected layer, and then aggregate the multi-level classification results through the classification averaging layer to output the final classification result.

[0015] In step (1), the MFCC feature extraction algorithm converts the one-dimensional time-domain signal into a two-dimensional Mel frequency cepstral coefficient feature matrix and outputs a spectral feature matrix with shape (n_frame, n_mfcc); where n_frame represents the number of time frames in the time dimension and n_mfcc represents the single-frame MFCC feature dimension in the frequency dimension.

[0016] In step (2), after feature dimension transformation, the output is a preprocessed feature matrix with shape (n_frame, n_hidden); where n_hidden is the hidden layer dimension.

[0017] In step (4), each Transformer layer performs temporal modeling of the input features using a multi-head self-attention mechanism and a feedforward neural network, with the input feature dimension being consistent with the hidden layer dimension. The output shape of the feature matrix of each Transformer layer is (n_frame, n_hidden). The feature matrix of each Transformer layer serves as the input to the next Transformer layer and as the input to the local feature enhancement branch. After processing by num_layer layers, a total of num_layer Transformer feature matrices are obtained.

[0018] In step (5), the DenseNet module enhances the local feature representation on the output of each Transformer layer, specifically including:

[0019] (51) For the feature matrix output by each Transformer layer, input it into each DenseNet module;

[0020] (52) The DenseNet module outputs a DenseNet feature matrix with shape (n_frame, n_hidden + dense_layer * growth_rate); where dense_layer is the number of internal layers of DenseNet and growth_rate is the growth rate parameter.

[0021] (53) The DenseNet feature matrix is ​​then passed through another DenseNet fully connected layer to compress the feature dimension back to n_hidden. Each layer yields a fully connected feature matrix with shape (n_frame, n_hidden).

[0022] In step (6), the average pooling layer converts the fully connected feature matrix of shape (n_frame, n_hidden) into a global feature vector of shape (1, n_hidden).

[0023] In step (7), the shape of the multi-scale feature matrix is ​​(num_layer, n_hidden).

[0024] In step (8), the binary classification fully connected layer maps the feature dimension to a classification matrix of shape (num_layer, 2); the final classification result has a shape of (1, 2).

[0025] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) By extracting features of different levels of Transformer and combining them with DenseNet for local feature extraction, and then classifying and fusing the features of each level, the global temporal modeling and local fine-grained feature enhancement of motor sound signals are unified, and multi-scale feature fusion is realized, thereby improving the accuracy of abnormal motor sound detection; (2) Based on the global modeling advantage of Transformer, the self-attention mechanism is used to effectively capture long-distance dependencies in the sound sequence and realize global context modeling; (3) Based on the local enhancement advantage of DenseNet, feature reuse is realized through dense connection structure, enhancing the local acoustic feature expression capability and improving the model's sensitivity to abnormal sound details; (4) Based on multi-scale fusion, the extraction and fusion of features of different levels are realized through multi-level Transformer-DenseNet parallel branches, taking into account both global and local information, and significantly improving the accuracy and robustness of abnormal motor sound detection; (5) The DenseNet module realizes the enhanced expression of local features on the output of each Transformer layer, making up for the shortcomings of Transformer in fine-grained local acoustic feature extraction. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0028] Example 1

[0029] This embodiment provides a specific implementation step for a motor abnormal sound detection method based on the combination of multi-layer feature aggregation Transformer and DenseNet. It uses joint modeling of Transformer and DenseNet to detect abnormal motor sounds, classifying motor sound signals into two categories (normal or abnormal). This model achieves global and local joint modeling of motor sound signals through modules such as MFCC acoustic feature extraction, Transformer global temporal modeling, DenseNet local feature enhancement, multi-scale feature pooling and fusion, and binary classification output. This addresses the shortcomings of single models in global temporal dependence or local fine-grained feature extraction, improving the accuracy and robustness of motor abnormal sound detection.

[0030] The flowchart of this method is as follows: Figure 1 As shown, the specific implementation steps include:

[0031] (1) Feature Extraction: In feature extraction, Mel Frequency Cepstral Coefficients (MFCC) is a commonly used sound feature extraction method. It extracts useful features that can represent audio signals by mimicking the human ear's perception of sounds at different frequencies. The specific steps are as follows: First, the time-domain signal of the motor sound during the operation of the industrial motor is collected as the model input and input to the MFCC feature extraction module to convert the one-dimensional time-domain signal into a two-dimensional Mel Frequency Cepstral Coefficients feature matrix; then, the number of frames n_frame in the Fourier transform of MFCC and the dimension n_mfcc of MFCC feature extraction are defined; finally, the time-domain motor sound data is processed by the MFCC feature extraction algorithm to output a spectral feature matrix of shape (n_frame, n_mfcc), where n_frame represents the number of time frames in the time dimension and n_mfcc represents the single-frame MFCC feature dimension in the frequency dimension.

[0032] (2) Preprocessing: Input the feature matrix of shape (n_frame, n_mfcc) into the preprocessing fully connected layer, perform feature dimension transformation, and output the preprocessed feature matrix of shape (n_frame, n_hidden), where n_hidden is the set uniform hidden layer dimension, which provides standardized feature input for subsequent Transformer modeling.

[0033] (3) Position encoding: The preprocessed feature matrix is ​​input into the sinusoidal position encoding module. The position information is added to the feature matrix through the sinusoidal function to obtain the preprocessed feature matrix with shape (n_frame, n_hidden) after position encoding, so as to make up for the defect that Transformer itself cannot perceive the temporal order.

[0034] (4) Global Temporal Modeling of Transformer: The feature matrix with added position encoding is input into the Transformer encoder composed of stacked num_layer layers to achieve global temporal modeling of the motor sound sequence. Each Transformer layer performs temporal modeling of the input features through a multi-head self-attention mechanism and a feedforward neural network, and the dimension of the input features is consistent with the dimension of the hidden layer. Therefore, the output shape of each layer is a Transformer feature matrix of (n_frame, n_hidden). This matrix serves as the input of the next Transformer layer on the one hand, and as the input of the local feature enhancement branch on the other hand, realizing multi-level feature reuse. After processing by num_layer layers, a total of num_layer Transformer feature matrices are obtained. Through the stacking of multiple Transformers, the model can effectively capture the long-distance dependencies in the motor sound sequence and learn global temporal context information.

[0035] (5) Multi-branch DenseNet local feature enhancement: The Transformer feature matrix output by each Transformer layer is input in parallel into the corresponding DenseNet module to enhance the local fine-grained acoustic feature representation capability:

[0036] (51) The feature matrix output by the i-th layer Transformer is input into the i-th layer DenseNet module;

[0037] (52) DenseNet achieves feature reuse and efficient transmission through a dense connection structure, and outputs a DenseNet feature matrix with shape (n_frame, n_hidden + dense_layer * growth_rate), where dense_layer is the number of internal layers of DenseNet and growth_rate is the growth rate parameter.

[0038] (53) The DenseNet feature matrix is ​​then passed through another DenseNet fully connected layer to compress the feature dimension back to n_hidden. Each layer yields a fully connected feature matrix with shape (n_frame, n_hidden).

[0039] Through the aforementioned parallel multi-branch structure, DenseNet achieves enhanced representation of local features in the output of each Transformer layer, making up for the Transformer's shortcomings in fine-grained local acoustic feature extraction.

[0040] (6) Average Pooling: The fully connected feature matrices output by each DenseNet fully connected layer are compressed temporally by an average pooling layer: the fully connected feature matrix of shape (n_frame, n_hidden) is converted into a global feature vector of shape (1, n_hidden). The pooling operation eliminates the temporal length difference and obtains a fixed-length feature representation.

[0041] (7) Multi-scale feature fusion: The global feature vectors after pooling of each layer are stacked layer by layer to obtain a multi-scale feature matrix with shape (num_layer, n_hidden), thereby realizing the fusion of global features at different levels.

[0042] (8) Binary classification: The multi-scale feature matrix is ​​input to the binary classification fully connected layer, which maps the feature dimension to a classification matrix with shape (num_layer, 2); then the classification averaging layer aggregates the multi-level classification results and outputs the final classification result with shape (1, 2), thus realizing the binary classification of motor sound signals as normal or abnormal.

[0043] Example 2

[0044] This embodiment provides a comparative experiment, as follows:

[0045] (1) Preparation of experimental data

[0046] Data source: Industrial test datasets are used. The motor component to be tested is placed in a test chamber, i.e., the data is collected in an environment with no background noise.

[0047] Data composition: The actual test dataset contains 658 samples of normal motor operation and 560 samples of abnormal motor operation.

[0048] Data partitioning: The training set contains 600 normal samples and 500 abnormal samples; the test set contains 58 normal samples and 60 abnormal samples; the training set data is used to train the model, and the test set data is used to compare the performance of the model.

[0049] (2) Parameter setting

[0050] The MFCC feature extraction parameters n_frame and n_mfcc are set to 5 and 32 respectively; the parameters set in the Transformer encoder are: num_heads=4, n_hideen=32, num_layers=3; in DenseNet: dense_layer=3, growth_rate=32; the learner Adam optimizer is set to a learning rate of 0.0001, the number of epochs is 500, and the batch size is 32.

[0051] (3) Experimental results and analysis

[0052] Based on the above preparations, a motor abnormal sound detection method based on the combination of multi-layer feature aggregation Transformer and DenseNet was compared with the Tansformer industrial motor equipment abnormal detection method. The accuracy of the motor abnormal sound detection method based on the combination of multi-layer feature aggregation Transformer and DenseNet was 94.9% (6 errors in 118 test sets), while the accuracy of the Tansformer industrial motor equipment abnormal detection method was 91.5% (10 errors in 118 test sets).

[0053] Therefore, the abnormal sound detection method for motors based on the combination of multi-layer feature aggregation Transformer and DenseNet can organically unify the global temporal modeling of motor sound signals with local fine-grained feature enhancement, thereby effectively improving the accuracy of abnormal sound detection.

Claims

1. A method for detecting abnormal motor sounds by combining feature aggregation Transformer and DenseNet, characterized in that, include: (1) Feature extraction: The time-domain signal of the motor sound is collected and processed by the MFCC feature extraction algorithm to obtain the spectrum feature matrix; (2) Preprocessing: Input the spectral feature matrix into the preprocessing fully connected layer to perform feature dimension transformation; (3) Position encoding: Input the preprocessed feature matrix into the sine position encoding module, and add position information to the feature matrix through a sine function; (4) Transformer global temporal modeling: Input the feature matrix with added position encoding into each Transformer layer to obtain the feature matrix of each Transformer layer; (5) Multi-branch DenseNet local feature enhancement: The feature matrix of each Transformer layer is input in parallel to the corresponding DenseNet module, and then the feature dimension is compressed through a DenseNet fully connected layer. (6) Average pooling: The fully connected feature matrices output by each DenseNet fully connected layer are compressed in temporal dimension by average pooling layers. (7) Multi-scale feature fusion: The global feature vectors after average pooling of each layer are stacked layer by layer to obtain a multi-scale feature matrix; (8) Classification: Input the multi-scale feature matrix into the binary classification fully connected layer, and then aggregate the multi-level classification results through the classification averaging layer to output the final classification result.

2. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (1), the MFCC feature extraction algorithm converts the one-dimensional time-domain signal into a two-dimensional Mel frequency cepstral coefficient feature matrix and outputs a spectral feature matrix with shape (n_frame, n_mfcc); where n_frame represents the number of time frames in the time dimension and n_mfcc represents the single-frame MFCC feature dimension in the frequency dimension.

3. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (2), after feature dimension transformation, the output is a preprocessed feature matrix with shape (n_frame, n_hidden); where n_hidden is the hidden layer dimension.

4. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (4), each Transformer performs temporal modeling of the input features through a multi-head self-attention mechanism and a feedforward neural network, and the dimension of the input features is consistent with the dimension of the hidden layer. The output shape of the feature matrix of each Transformer is (n_frame, n_hidden).

5. The method for detecting abnormal motor sounds according to claim 4, characterized in that, Each Transformer feature matrix is ​​used as the input to the next Transformer layer and as the input to the local feature enhancement branch. After processing by num_layer layers, a total of num_layer Transformer feature matrices are obtained.

6. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (5), the DenseNet module enhances the local feature representation on the output of each Transformer layer, specifically including: (51) For the feature matrix output by each Transformer layer, input it into each DenseNet module; (52) The DenseNet module outputs a DenseNet feature matrix with shape (n_frame, n_hidden + dense_layer * growth_rate); where dense_layer is the number of internal layers of DenseNet and growth_rate is the growth rate parameter. (53) The DenseNet feature matrix is ​​then passed through another DenseNet fully connected layer to compress the feature dimension back to n_hidden. Each layer yields a fully connected feature matrix with shape (n_frame, n_hidden).

7. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (6), the average pooling layer converts the fully connected feature matrix of shape (n_frame, n_hidden) into a global feature vector of shape (1, n_hidden).

8. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (7), the shape of the multi-scale feature matrix is ​​(num_layer, n_hidden).

9. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (8), the binary classification fully connected layer maps the feature dimension to a classification matrix of shape (num_layer, 2).

10. The method for detecting abnormal motor sounds according to claim 1, characterized in that, In step (8), the final classification result has a shape of (1, 2).

Images