An unsupervised method for abnormal sound detection of machine equipment based on domain generalization technology

Abstract

The present invention discloses an unsupervised method for detecting abnormal sounds of machine equipment based on domain generalization technology, comprising: collecting normal audio data and fault audio data emitted by the operation of the machine equipment, dividing them into a source domain dataset and a target domain dataset; converting the original waveform of the audio data into a log-mel spectrogram and an amplitude spectrogram; inputting the log-mel spectrogram into a log-mel spectrogram network to extract embedded features, and inputting the amplitude spectrogram into an amplitude spectrogram network to extract embedded features, connecting the embedded features extracted from the two networks, adding a coordinate attention mechanism module, and using joint training to distinguish machine types; using a Gaussian mixture model with a complete covariance matrix to calculate anomaly scores; and inputting the source domain dataset and the target domain dataset into a domain generalization unsupervised network fault detection model to complete domain adaptive mechanical fault detection. The method for detecting abnormal sounds of machine equipment based on domain generalization technology provided by the present invention can effectively solve the domain shift problem existing in current machine anomaly detection.

Description

Technical Field

[0001] The present invention belongs to the field of fault detection, and in particular relates to an unsupervised abnormal sound detection method for machine equipment based on domain generalization technology. Background Art

[0002] Identifying machine sounds during operation and promptly detecting machine anomalies is crucial for machine status monitoring. In recent years, with the rise of industrial big data analytics and the rapid development of computing power, data-driven methods have become an active research area in machine fault detection. Among the many data-driven fault detection methods, deep learning-based methods have garnered widespread attention due to their high accuracy, fast response, and robust security.

[0003] Most deep learning-based fault detection methods achieve good results when the training and test data follow the same characteristic distribution. However, in real-world situations, acoustic features vary with operating conditions, and the assumption that training and test data follow the same distribution rarely holds true in practical applications. The distributional differences between the source domain represented by the training data and the target domain represented by the test data often lead to reduced reliability of deep learning-based fault detection models, hindering the application of deep learning-based fault detection methods in real-world projects.

[0004] To improve the robustness and adaptability of deep learning-based fault detection models, research on domain adaptation technology has attracted considerable attention. Fault detection methods based on domain adaptation bridge the gap between domains by adjusting the distribution of the source and target domains in a high-level subspace. While these methods offer improved performance over traditional deep learning-based methods, they still face significant challenges in addressing practical cross-domain fault detection tasks. Unlabeled target data is crucial for facilitating the transfer of detection knowledge from the source domain to the target domain, but this unlabeled data is difficult to collect in advance in real industrial scenarios. Furthermore, fault detection methods based on domain adaptation only generalize detection knowledge from a single source domain to a specific target domain. Therefore, when the operating conditions of the samples to be detected change, these methods inevitably overfit to a single domain, impacting fault detection accuracy.

[0005] To overcome data's reliance on labels and single features and generalize fault detection models to unknown operating conditions, domain generalization technology has become a hot research topic in intelligent condition monitoring and fault detection for machinery and equipment. Abnormal sound detection methods based on domain generalization learn common acoustic features from various related domains, transfer these features to unknown operating conditions, and effectively perform target detection tasks.

[0006] The abnormal sound detection framework based on domain generalization technology can better achieve real-time cross-domain fault detection without assuming the availability of target domain data. That is, it can directly learn a generalized detection model from source domain data and effectively solve target domain tasks with different characteristics.

[0007] However, most existing abnormal sound detection methods based on domain generalization techniques are unable to detect unknown fault modes. When the target domain data comes from unknown operating conditions and unknown fault modes appear during the testing phase, the detection model may misdetect the unknown fault mode, affecting its reliability. Summary of the Invention

[0008] In view of this, the present invention proposes an unsupervised abnormal sound detection method for machine equipment based on domain generalization technology, comprising the following steps:

[0009] S1. Collect audio data emitted by mechanical equipment during operation, including normal audio data and fault audio data. For each type of mechanical equipment, m1 normal audio clips, n1 fault audio clips, and label information are used as the source domain dataset, and m2 normal audio clips and n2 fault audio clips of each type of mechanical equipment are used as the target domain dataset.

[0010] S2, converting the original waveform of the audio data into a log-mel spectrogram and an amplitude spectrogram;

[0011] S3. Construct a domain generalization unsupervised network-based fault detection model, wherein the fault detection model extracts an embedded feature network composed of a log-mel spectrogram network and an amplitude spectrogram network;

[0012] S4. Input the log-mel spectrogram into the log-mel spectrogram network to extract embedded features, and input the amplitude spectrogram into the amplitude spectrogram network to extract embedded features. The embedded features extracted by the two networks are concatenated, and a coordinate attention mechanism module is added to focus on learning important information in the features and suppress unimportant information. The sub-cluster AdaCos loss function is used to jointly train the domain generalization unsupervised network fault detection model to distinguish machine types and different attribute information of machine types;

[0013] S5. Input the embedded features output from the coordinate attention mechanism into the Gaussian mixture model with the complete covariance matrix to calculate the cosine similarity between the test sample and the corresponding class center, and use the log-likelihood value of the result as the anomaly score;

[0014] S6. Input the source domain dataset and the target domain dataset into the domain generalization unsupervised network fault detection model to complete domain adaptive mechanical fault detection.

[0015] Furthermore, in step S2, the log-mel spectrogram is normalized by sampling time mean.

[0016] Furthermore, the log-mel spectrogram network is based on an improved ResNet architecture, including a 2D convolution layer, four improved residual blocks and a maximum pooling; the amplitude spectrogram network consists of 3 one-dimensional convolutions and 5 dense layers.

[0017] Furthermore, in step S4, the sub-cluster AdaCos loss function is used to jointly train the model. The sub-cluster AdaCos loss function learns multiple sub-clusters for each machine type. For a given sample x i , the probability that the subcluster belongs to the jth type in the Cth type:

[0018]

[0019] Where M (j) represents all subclusters belonging to class j, S represents the number of subclusters belonging to class j, represents the adaptive scale parameter, which is automatically adjusted according to the machine type, CS represents the total number of subclusters, and θ i,k ∈[0,π] represents the angle defined by cosine similarity Define a learned cluster center W k ,θ i,l ∈[0,π] represents the angle defined by cosine similarity Define a learned subcluster W l .

[0020] Furthermore, the Gaussian mixture model in step S5 is implemented using scikit-learn, and the anomaly score is obtained by training the embedding of the Gaussian mixture model and using negative weighted log-likelihood.

[0021] Furthermore, a domain mixing approach is adopted to train the domain generalization-based unsupervised network fault detection network model. For each combination of mechanical equipment types, each embedding corresponding to the regular training samples of the source domain is mixed with the mean of random samples of the target domain.

[0022] Furthermore, the optimized linear interpolation method is used to solve the mixed parameters of multiple fault types in the dataset R composed of source domain and target domain samples, where two samples x i ,x j ∈R and the corresponding class label y i , j ∈[0,1];

[0023] x mix =x x +(1-)x j

[0024] y mix =y i +(1-)y j

[0025] Where the mixing coefficient λ∈[0,1], x mix Represents the obtained mixed sample, y mix Represents the class label of the obtained mixed sample.

[0026] The beneficial effects brought about by the technical solution provided by the present invention are:

[0027] The machine abnormal sound detection method based on domain generalization technology provided by the present invention can effectively solve the domain shift problem existing in the current machine abnormality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0029] Figure 2 It is a block diagram of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0031] refer to Figure 1 and Figure 2 , Figure 1 is a flow chart of a method according to an embodiment of the present invention; Figure 2 It is a block diagram of a method according to an embodiment of the present invention.

[0032] An unsupervised method for detecting abnormal sounds of machine equipment based on domain generalization technology according to an embodiment of the present invention includes the following steps:

[0033] S1. Collect audio data from the operation of mechanical equipment, including normal audio data and faulty audio data. For each type of mechanical equipment, m1 normal audio clips, n1 faulty audio clips, and their labels are used as the source domain dataset. For each type of mechanical equipment, m2 normal audio clips and n2 faulty audio clips are used as the target domain dataset. The sound clips in the source and target domain datasets are not exactly the same.

[0034] In this example, we prepared audio data from various types of machinery operating in real industrial scenarios. For each machine type, we used 900 normal sound clips and 50 abnormal sound clips, along with their labels, as the source domain dataset. For each machine type, we used 500 normal sound clips and 500 abnormal sound clips as the target domain dataset. Each audio segment in both the source and target domain datasets was 10 seconds long and sampled at 16 kHz.

[0035] The source domain dataset and the target domain dataset include multiple machine types, consisting of multiple sound clips of various machine types such as "bearing", "fan", "gearbox", "pulley", "valve" and "coal mill".

[0036] To determine whether a given test sample is normal, a single decision threshold for each domain is calculated by taking the 90th percentile of the log-likelihood values of the corresponding normal training samples, and then all anomaly scores of test samples belonging to the same domain and above this threshold are marked as anomalous.

[0037] S2. Convert the original waveform of the audio data into a log-mel spectrogram and an amplitude spectrogram.

[0038] The original audio is processed into a log-mel spectrogram with a Hamming window length of 1024 and a step size of 256 and 128, and an amplitude spectrogram of the entire audio signal, and the log-mel spectrogram is normalized by the sampling time mean.

[0039] S3. Construct a domain generalization unsupervised network fault detection model, wherein the fault detection model extracts an embedded feature network composed of a log-mel spectrogram network and an amplitude spectrogram network.

[0040] In this embodiment, the log-mel spectrogram network is based on an improved ResNet architecture, including a 2D convolution layer, four improved residual blocks and a maximum pooling layer; the magnitude spectrogram network consists of three one-dimensional convolutional layers and five dense layers.

[0041] S4: Input the log-mel spectrogram into the log-mel spectrogram network to extract embedded features, and the amplitude spectrogram into the amplitude spectrogram network to extract embedded features. The embedded features extracted from the two networks are concatenated, and a coordinate attention mechanism module is added to focus on learning important information in the features and suppress unimportant information. The domain generalization unsupervised network fault detection model is jointly trained using the sub-cluster AdaCos loss function to distinguish between machine types and different attribute information of machine types. When retraining the system, instead of repeating the integration process using a single sub-cluster AdaCos loss between machine types, the same number of sub-cluster AdaCos losses are used.

[0042] The sub-cluster AdaCos loss function is used to jointly train the domain generalization unsupervised network fault detection model. The sub-cluster AdaCos loss function learns multiple sub-clusters for each machine type. For a given sample x i , the probability that the subcluster belongs to the jth type in the Cth type:

[0043]

[0044] Where M (j) represents all subclusters belonging to class j, S represents the number of subclusters belonging to class j, represents the adaptive scale parameter, which can be automatically adjusted according to the machine type; CS represents the total number of subclusters; θ i,k ∈[0,π] represents the angle defined by cosine similarity Define a learned cluster center W k θ i,l ∈[0,π] represents the angle defined by cosine similarity Define a learned subcluster W l .

[0045] The domain generalization-based unsupervised network fault detection network model is trained using a domain mixing approach. For each combination of mechanical equipment types, each embedding corresponding to a regular training sample in the source domain is mixed with the mean of a random sample in the target domain.

[0046] S5. Input the embedded features output from the coordinate attention mechanism into the Gaussian mixture model with a complete covariance matrix to calculate the cosine similarity between the test sample and the corresponding class center, and use the log-likelihood value of the result as the anomaly score.

[0047] The Gaussian mixture model is implemented using scikit-learn. The average value of the sub-cluster AdaCos loss is initialized and 10 is added to the diagonal. -3 A regularized covariance matrix is obtained. The anomaly score is obtained by training the embedding of the Gaussian mixture model and using the negative weighted log-likelihood.

[0048] S6. Input the source domain dataset and the target domain dataset into the domain generalized unsupervised network fault detection model.

[0049] The optimized linear interpolation method is used to solve the mixed parameters of multiple fault types in the dataset R composed of source domain and target domain samples, where two samples x i ,x j ∈R and the corresponding class label y i ,y j ∈[0,1];

[0050] x mix =λx i +(1-λ)x j

[0051] y mix =λy i +(1-λ)y j

[0052] Where the mixing coefficient λ∈[0,1], x mixRepresents the obtained mixed sample, y mix Represents the class label of the obtained mixed sample.

[0053] The model learns mechanical fault feature information from the source domain dataset and tests it on the target domain dataset to complete domain-adaptive mechanical fault detection.

[0054] In this example, the effectiveness of the method was evaluated using the area under the receiver operating characteristic curve (ROC) (AUC) and partial AUC (pAUC). For the tests performed by this method, pAUC can be regarded as the AUC calculated over the low false positive rate (FPR) range [0, p].

[0055] In specific experiments, the performance of the fault detection system proposed by this method is close to the performance obtained in the source domain in most cases. The AUC and pAUC scores of the fault detection system proposed by this method for the machine type "valve" in the target domain are 99.24% and 96.42%.

[0056] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An unsupervised method for detecting abnormal sound of machine equipment based on domain generalization technology, characterized in that: The following steps are involved: S1. Collect audio data emitted by mechanical equipment during operation, including normal audio data and fault audio data. For each type of mechanical equipment, m1 normal audio clips, n1 fault audio clips, and label information are used as the source domain dataset, and m2 normal audio clips and n2 fault audio clips of each type of mechanical equipment are used as the target domain dataset. S2, converting the original waveform of the audio data into a log-mel spectrogram and an amplitude spectrogram; S3. Construct a domain generalization unsupervised network-based fault detection model, wherein the fault detection model extracts an embedded feature network composed of a log-mel spectrogram network and an amplitude spectrogram network; S4. Input the log-mel spectrogram into the log-mel spectrogram network to extract embedded features, and input the amplitude spectrogram into the amplitude spectrogram network to extract embedded features. The embedded features extracted by the two networks are concatenated, and a coordinate attention mechanism module is added to focus on learning important information in the features and suppress unimportant information. The sub-cluster AdaCos loss function is used to jointly train the domain generalization unsupervised network fault detection model to distinguish machine types and different attribute information of machine types; S5. Input the embedded features output from the coordinate attention mechanism into the Gaussian mixture model with the complete covariance matrix to calculate the cosine similarity between the test sample and the corresponding class center, and use the log-likelihood value of the result as the anomaly score; S6. Input the source domain dataset and the target domain dataset into the domain generalization unsupervised network fault detection model to complete domain adaptive mechanical fault detection.

2. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: In step S2, the log-mel spectrogram is normalized by the sampling time mean.

3. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: In step S3, the log-mel spectrogram network is based on an improved ResNet architecture, including a 2D convolution layer, four improved residual blocks and a maximum pooling layer; the amplitude spectrogram network consists of three one-dimensional convolutional layers and five dense layers.

4. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: In step S4, the sub-cluster AdaCos loss function is used to jointly train the domain generalization unsupervised network fault detection model. The sub-cluster AdaCos loss function learns multiple sub-clusters for each machine type. For a given sample x i , the probability that a subcluster belongs to class j in a type: Where M (j) represents all subclusters belonging to class j, S represents the number of subclusters belonging to class j, represents the adaptive scale parameter, C represents the number of sub-cluster types, CS represents the total number of sub-clusters, θ i,k ∈[0,π] represents the angle defined by cosine similarity Define a learned cluster center W k ,θ i,l ∈[0,π] represents the angle defined by cosine similarity Define a learned subcluster W l .

5. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: The Gaussian mixture model in step S5 is implemented using scikit-learn. The embedding of the Gaussian mixture model is trained and the anomaly score is obtained using the negative weighted log-likelihood.

6. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: The domain generalization-based unsupervised network fault detection network model is trained using a domain mixing approach. For each combination of mechanical equipment types, each embedding corresponding to a regular training sample in the source domain is mixed with the mean of a random sample in the target domain.

7. The unsupervised abnormal sound detection method for machine equipment based on domain generalization technology according to claim 1 is characterized in that: The optimized linear interpolation method is used to solve the mixed parameters of multiple fault types in the dataset R composed of source domain and target domain samples, where two samples x i ,x j ∈R and the corresponding class label y i ,y j ∈[0,1]; x mix =λx i +(1-λ)x j and mix =λy i +(1-λ)y j Where the mixing coefficient λ∈[0,1], x mix Represents the obtained mixed sample, y mix Represents the class label of the obtained mixed sample.

Images