An unsupervised machine abnormal sound detection method based on lightweight network

Abstract

The present application is directed to the feature combination problem of abnormal sound detection in industrial scenarios, and proposes an unsupervised machine abnormal sound detection method based on a lightweight network. The method uses log-Mel spectrogram and SincNet spectrogram fusion features as input, making the feature information of machine working sound more rich, which is beneficial to improve the detection performance of the lightweight network. The detection model uses a more lightweight ES attention mechanism in the MobileNetV3 backbone network, which can not only avoid the risk of fine feature loss in the pooling process, but also further reduce the computational complexity of the model, so that the system can be better deployed on mobile devices, enhancing the practicality and flexibility of machine abnormal sound detection. The method is not only suitable for abnormal detection of machine working sound signals in industrial scenarios, but also suitable for mobile deployment of various abnormal sound detection tasks.

Description

Technical Field

[0001] This invention relates to the field of abnormal machine sound detection technology, and in particular to an unsupervised method for detecting abnormal machine sounds based on lightweight networks. Background Technology

[0002] Machine abnormal sound detection technology belongs to the research field of sound signal detection and has important application value in scenarios such as equipment operation status monitoring and automated product quality inspection.

[0003] In practical industrial applications, factory machinery is mostly composed of components such as pumps, valves, fans, and motors. Monitoring the anomalies of these critical machine parts plays a vital role in the stable operation of production machinery, enterprise production efficiency, product quality, and the safety of employees and property. Currently, the mainstream method for monitoring the factory production environment is through video surveillance. However, video stream processing requires significant storage and computing power, making the hardware cost of such systems very expensive and hindering their adoption by small and medium-sized enterprises with limited purchasing power. Sound-based monitoring systems, as an important component of safety monitoring, can determine whether machines are malfunctioning based on the sounds they emit. This system can promptly detect abnormal machine conditions, providing customers with data-driven decision-making support, thereby preventing major accidents and emergencies and playing a positive role in ensuring enterprise production safety.

[0004] Currently, the detection of abnormal machine sounds is mostly accomplished through manual inspection and traditional deep learning methods, which are slow and inefficient. While large-scale network models can achieve better detection accuracy, their large number of parameters and high computational complexity lead to prohibitively high hardware costs, making it difficult to deploy such algorithms directly on mobile devices. In practical abnormal sound detection applications, the differences between abnormal and normal machine sounds are sometimes extremely subtle, and existing lightweight network models are insufficient to extract and analyze such minute features from the characteristics of the machine's sound signals during operation. Summary of the Invention

[0005] To address the shortcomings of existing technologies and meet the lightweight requirements of abnormal sound monitoring on factory mobile devices, this invention proposes an unsupervised machine abnormal sound detection method based on a lightweight network. While ensuring the accuracy of abnormal sound detection, it can achieve low power consumption and computational efficiency in abnormal sound monitoring.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An unsupervised machine anomalous sound detection method based on lightweight networks includes the following steps:

[0008] S1. For the machine under test in its working state, collect audio data of normal and abnormal types of machines, divide the collected audio data, and generate training and test sets.

[0009] The training set contains only normal audio samples when the machine is working, while the test set contains both normal and abnormal audio samples when the machine is working.

[0010] S2. Extract log-Mel spectrogram features and SincNet spectrogram features from the audio data samples in the training set, and fuse the log-Mel spectrogram and SincNet spectrogram to obtain the fused feature IMS spectrogram.

[0011] S3. Based on the MobileNetV3 network, construct an improved lightweight ES-MobileNetV3 network. Specifically, modify the global average pooling layer of the Efficient Channel Attention (ECA) network to a soft pooling layer to obtain the ES attention module. Name the modified ECA attention network the ES attention network.

[0012] Then, the Squeeze-and-Excitation Networks (SE) in the MobileNetV3 network were replaced with the ES attention network to obtain the lighter detection model ES-MobileNetV3;

[0013] The training set of S2 is input into the improved lightweight ES-MobileNetV3 network for model training to obtain the trained lightweight ES-MobileNetV3 network.

[0014] S4. Input the test set into the lightweight ES-MobileNetV3 network trained in S3 to detect whether the machine's operating sound is abnormal.

[0015] Compared with existing technologies, the detection method of this invention can achieve the following beneficial effects:

[0016] This invention's method for detecting abnormal machine sounds first uses a fusion of log-Mel and SincNet spectrograms as audio input. Then, it learns machine operating audio information through a lightweight ES-MobileNetV3 network model. This effectively improves the inference speed of the detection model, reduces the number of model parameters, and enhances the robustness of the detection system. The log-Mel spectrogram focuses on feature extraction in the mid-to-low frequency domain, while the SincNet spectrogram focuses on extracting harmonic and impulse components related to the machine sound signal. This feature fusion method effectively improves the representation of feature information in the machine operating audio, providing richer features for the subsequent anomaly detection model. The ES attention module in the ES-MobileNetV3 network not only effectively reduces the model's computational load but also retains more audio information with prominent energy signals, making it more suitable for deployment on mobile devices and improving the flexibility and portability of the abnormal machine sound detection system.

[0017] The abnormal machine sound detection method of the present invention is not only applicable to the abnormal detection of machine working sound signals in industrial scenarios, but also applicable to mobile terminal deployment for various abnormal sound detection tasks. Attached Figure Description

[0018] Figure 1 This is a flowchart of the detection method in the embodiment;

[0019] Figure 2 This is a flowchart illustrating the soft pooling method calculation in the embodiment;

[0020] Figure 3 This is a structural diagram of the ES attention module in the embodiment;

[0021] Figure 4 This is a diagram of the ES-MobileNetV3 network structure in the embodiment. Detailed Implementation

[0022] The present invention will be further described below with reference to embodiments and accompanying drawings, but this is not intended to limit the scope of the invention.

[0023] Example

[0024] An unsupervised machine anomalous sound detection method based on lightweight networks, such as Figure 1 As shown, it includes the following steps:

[0025] S1. A dataset is constructed by collecting audio data from real factory machines during operation. The collected audio data is then divided into training and testing sets. The training set contains only normal audio samples from when the machines are operating, while the testing set includes both normal and abnormal audio samples from when the machines are operating. The machine types in the dataset include "valves," "fans," "slide rails," and "water pumps," etc. The training set for each machine type consists of 2500 audio samples, and the testing set consists of 100 normal samples and 100 abnormal samples. Each audio sample in both the training and testing sets has a duration of 10 seconds and a sampling frequency of 16kHz.

[0026] S2. Extract log-Mel spectrogram features and SincNet spectrogram features from the audio data samples in the training set. The log-Mel spectrogram has a frame size of 1024, a jump length of 512, and 128 Mel filters. The SincNet convolutional layer of the SincNet spectrogram consists of 128 SincNet filters with a length of L = 251 (i.e., each filter contains 251 coefficients). Then, the output of each filter is normalized using adaptive average pooling.

[0027] S3. Based on the MobileNetV3 network, an improved lightweight ES-MobileNetV3 network is constructed. Specifically, the global average pooling layer of the efficient channel attention ECA network is modified to a soft pooling layer, and the modified ECA attention network is named the ES attention network.

[0028] Then, the compression and activation SE network in the MobileNetV3 network is replaced with the ES attention network to obtain a lighter detection model, ES-MobileNetV3.

[0029] The training set of S2 is input into the ES-MobileNetV3 network for model training, resulting in a trained lightweight ES-MobileNetV3 network.

[0030] S4. Input the test set into the lightweight ES-MobileNetV3 network trained in S3 to detect whether the machine's operating sound is abnormal.

[0031] The lightweight network model MobileNetV3 introduces an inverted residual structure with a linear bottleneck to improve the efficiency of the layer structure by leveraging the low-rank nature of the problem. This network structure further reduces the number of channel-level bneck operations, improving detection speed while slightly reducing network detection accuracy, making it more suitable for deployment on lightweight computing devices. Building on this, this invention proposes a novel activation function, H-Swish, to enhance feature extraction accuracy and improve detection speed. Furthermore, a squeezed and activated SE attention network is introduced into the inverted residual structure. The SE attention network first performs global average pooling on the features of each channel, then uses two fully connected layers to capture the interactions between channels, improving the model's feature representation capability to some extent. However, due to the extremely high computational cost of fully connected layers, the SE attention network performs dimensionality reduction on the feature matrix at the fully connected layers, followed by dimensionality increase to obtain the importance of the weights. This dimensionality reduction process at the fully connected layers results in the loss of some feature information, negatively impacting the network model's detection performance.

[0032] To reduce information loss caused by dimensionality reduction, this embodiment uses a non-dimensionality-reduced local cross-channel interactive ES attention network instead of the SE attention network, further achieving a lightweight network model. Avoiding dimensionality reduction in the network model is crucial for learning channel attention. By capturing local channel information through modules, the importance of each feature channel can be obtained. Appropriate cross-channel interaction can significantly reduce model complexity while maintaining excellent performance and suppressing some feature information that is useless to the current task.

[0033] To address the issue that the global average pooling layer used in the network model during machine audio signal processing weakens the feature representation of audio data, the ES attention module can be obtained by modifying the global average pooling of the ECA attention module to soft pooling.

[0034] The computational process of the soft pooling method, such as... Figure 2 As shown, using soft pooling instead of global average pooling can suppress the display of the entire feature region to a certain extent, while retaining some prominent features, preserving more refined feature map information as much as possible, and enhancing the difference in feature information between abnormal machine audio and normal audio.

[0035] The calculation formula for soft pooling is as follows:

[0036]

[0037] In the formula, This represents the output after soft pooling, where R is the pooling region, i, j are the indices of the pooling region, and w i Indicates the activation weight, a i and a jThis represents the activation value of a point within the pooling region;

[0038] In this embodiment, the pooling region R is 4, and the soft pooling method is calculated as follows:

[0039]

[0040] In this embodiment, the pooling region R is 4. The soft pooling method calculation process is as follows: First, calculate the feature weights corresponding to the 4 activation values ​​in the pooling region R respectively. Then, perform a weighted sum of the 4 activation values ​​and their corresponding feature weights to obtain the output value of the 4 feature points in the pooling region R after soft pooling.

[0041] ES attention module structure diagram, such as Figure 3 As shown, firstly, the fused feature IMS spectral features are aggregated using soft pooling to obtain global channel information. Then, the convolution kernel size k is adaptively calculated using the channel dimension C. The adaptive function calculation formula is as follows:

[0042]

[0043] In the formula, C represents the channel dimension, and γ and b are user-defined parameters, set to 2 and 1 respectively; |t| odd Represents the odd number closest to t;

[0044] Finally, the channel weights are obtained through the Sigmoid function, and the generated channel weights are multiplied element-wise with the input features to obtain the attention-enabled features.

[0045] The detection model learns the feature information of the machine when it is working normally on the training set. After obtaining the trained ES-MobileNetV3 network model, it is tested on the test set to complete the detection of abnormal machine sounds.

[0046] The main structure of the ES-MobileNetV3 network model, such as Figure 4 As shown, the ES-MobileNetV3 model replaces the SE attention network with the ES attention network on top of the MobileNetV3 network. This preserves more feature information of the acoustic signal, and the soft pooling operation effectively retains the fine features of the machine sound, enhancing the recognizability of feature information. It also effectively reduces the computational complexity of the model without sacrificing accuracy.

[0047] In this embodiment, the area under the receiver operating characteristic (AUC) curve and partial AUC (pAUC) are used to evaluate the model's detection performance. AUC is a common anomaly detection performance metric, representing the probability that the model predicts a positive sample as positive when randomly selecting a positive sample and a negative sample from the test set, is more likely to predict a negative sample as positive. Furthermore, if a machine anomaly detection model frequently identifies normal sounds as abnormal, thus issuing false alarms, then the model's reliability is insufficient. pAUC can be used to comprehensively evaluate the model's reliability; pAUC can be considered as the AUC calculated over a low false positive rate (FPR) range [0, p]. In this embodiment, p is set to 0.1.

[0048] In the embodiments, the abnormal machine sound detection method proposed in this invention is used. The AUC and pAUC of the slide rail machine type can reach 99.42% and 98.67%, respectively. The results show that the abnormal machine sound detection system proposed in this method can achieve excellent results in most cases. This research on the detection of machine operating sounds such as slide rails will play an important role in the monitoring of the operating status of parts in future factory production lines.

Claims

1. An unsupervised machine anomalous sound detection method based on lightweight networks, characterized in that, Includes the following steps: S1. For the machine under test in the working state, collect audio data of normal and abnormal types of machines, divide the collected audio data, and generate training set and test set. The training set contains only normal audio samples when the machine is working, while the test set contains both normal and abnormal audio samples when the machine is working. S2. Extract log-Mel spectrogram features and SincNet spectrogram features from the audio data samples in the training set, and fuse the log-Mel spectrogram and SincNet spectrogram to obtain the fused feature IMS spectrogram. S3. Based on the MobileNetV3 network, an improved lightweight ES-MobileNetV3 network is constructed. Specifically, the global average pooling layer of the efficient channel attention ECA network is modified to a soft pooling layer to obtain the ES attention module. The modified ECA attention network is named the ES attention network. The calculation formula for soft pooling is as follows: In the formula, This represents the output result after soft pooling. For pooling regions, The index subscript for the pooled region. Indicates activation weight, and This represents the activation value of a point within the pooling region; The ES attention module first aggregates the fused feature IMS spectral features through soft pooling to obtain global channel information, and then through the channel dimension. Adaptive computation of convolution kernel size The adaptive function is calculated as follows: In the formula, For the channel dimension, and These are custom parameters, set to 2 and 1 respectively; Indicates distance The most recent odd number; Finally, the channel weights are obtained through the Sigmoid function, and the generated channel weights are multiplied element-wise with the input features to obtain the attention-enabled features. Then, the compression and activation SE network in the MobileNetV3 network is replaced with the ES attention network to obtain a lighter detection model, ES-MobileNetV3. Input the training set of S2 into the ES-MobileNetV3 network for model training to obtain a trained lightweight ES-MobileNetV3 network. S4. Input the test set into the lightweight ES-MobileNetV3 network trained in S3 to detect whether the machine's operating sound is abnormal.

2. The unsupervised machine abnormal sound detection method according to claim 1, characterized in that: The training set for machine type in S1 consists of 2500 audio samples, and the test set consists of 100 normal samples and 100 abnormal samples. The duration of each audio sample in both the training and test sets is 10 seconds, and the sampled audio is 16kHz.

3. The unsupervised machine abnormal sound detection method according to claim 1, characterized in that: The frame size of the log-Mel spectrum in S2 is 1024, the jump length is 512, the number of Mel filters is 128, and the SincNet convolutional layer of the SincNet spectrum consists of 128 SincNet filters with a length of L=251. The output of each filter is then normalized using adaptive average pooling.

Images