Non-supervision abnormal sound detection method and system for mechanical equipment based on domain generalization

Abstract

The invention provides a domain generalization-based unsupervised abnormal sound detection method and system for mechanical equipment, and the method comprises the steps: constructing an SE-ECA fusion attention mechanism module based on a global context, constructing an MFCC branch network architecture and a Log-Mel branch network architecture, adding the modules to each branch network architecture, and carrying out the detection of the abnormal sound of the mechanical equipment. Audio feature extraction is carried out on the MFCC spectrogram features and the Log-Mel spectrogram features respectively; embedding the audio features extracted by the two branch network architectures into a channel dimension for merging to form a comprehensive feature vector, and classifying the comprehensive feature vector; an anomaly detection model is constructed and trained, the anomaly detection model uses a Gaussian mixture model to calculate a global anomaly score, then uses a local anomaly factor to detect local anomaly, and then performs weighted averaging, so that normal sound features can be automatically learned, anomaly can be detected, and the anomaly detection performance of equipment is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial equipment monitoring and fault diagnosis, and in particular relates to an unsupervised abnormal sound detection method and system for mechanical equipment based on domain generalization. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of automated factories, machines and equipment are becoming more and more complex and sophisticated, and the types of faults are also increasing. Traditional manual detection methods are inefficient and costly. The development of machine learning and deep learning technologies has made it possible to automatically detect machine anomalies. The machine abnormal sound detection method based on deep learning can make full use of the advantages of sound signals to achieve rapid response to machine failures. However, the probability of machine failure in actual factory production is relatively small, and it is difficult to systematically collect abnormal sound data on a large scale. The types of faults and abnormal sound patterns are also relatively complex, making it difficult to design and develop a data set covering all abnormal situations for supervised training. Due to the lack of abnormal supervision signals, it is still a challenge to achieve high-performance machine abnormal sound detection by training models only with normal sound data.

[0004] At the same time, the domain shift problem is also a major problem in practical applications. Domain shift refers to the difference between training and testing conditions, including the operating state of the machine (such as speed, load, and temperature) or environmental conditions (such as noise type, noise level, and microphone position). Changes in these conditions will lead to changes in data distribution, thereby reducing the anomaly detection performance of mechanical equipment. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes an unsupervised abnormal sound detection method and system for mechanical equipment based on domain generalization. The present invention can automatically learn normal sound features and detect anomalies, thereby improving the abnormality detection performance of the equipment.

[0006] According to some embodiments, the present invention adopts the following technical solutions:

[0007] A method for detecting abnormal sound of mechanical equipment without supervision based on domain generalization comprises the following steps:

[0008] Acquire audio data emitted by different mechanical equipment, including normal audio data and abnormal audio data, divide the collected audio data into training data sets and test data sets;

[0009] Extract MFCC spectrogram features and Log-Mel spectrogram features from the audio data samples in the training set;

[0010] Constructing a global context-based SE-ECA fusion attention mechanism module, respectively constructing an MFCC branch network architecture and a Log-Mel branch network architecture, adding the module to each branch network architecture, and respectively using the branch network architecture after adding the module to extract and process audio features of the MFCC spectrogram features and the Log-Mel spectrogram features;

[0011] The audio feature embeddings extracted by the two branch network architectures are merged in the channel dimension to form a comprehensive feature vector, the comprehensive feature vector is classified, and the classification results are stacked with the true labels to form the final classification output;

[0012] Constructing and training an anomaly detection model, wherein the anomaly detection model uses a Gaussian mixture model to calculate a global anomaly score, uses a local anomaly factor to detect local anomalies, and performs a weighted average of the global anomaly score and the local anomaly score;

[0013] The trained anomaly detection model is used to evaluate the test dataset and the log-likelihood value of each test sample is calculated and used as the final anomaly score.

[0014] As an optional implementation, the process of extracting MFCC spectrogram features from audio data samples in a training set includes performing a short-time Fourier transform on the original audio signal to obtain a spectrogram, converting the spectrogram into an energy spectrum on a Mel-frequency scale, taking its logarithm, and finally obtaining the MFCC features through a discrete transform.

[0015] As an optional implementation, the process of extracting Log-Mel spectrogram features from audio data samples in a training set includes performing a short-time Fourier transform on the original audio signal to calculate the amplitude spectrum, mapping it to a Mel frequency scale, and then performing a logarithmic transform to obtain a log-Mel spectrogram.

[0016] As an optional implementation, the global context-based SE-ECA fusion attention mechanism module combines the two attention mechanisms of SE and ECA, obtains the global information of the feature map through global pooling and maximum pooling, and splices the global information obtained by global pooling and maximum pooling in the channel dimension to form a context vector;

[0017] The concatenated features are processed by two fully connected layers to obtain the input feature map, which is weighted by the SE mechanism to adjust the feature dimensions between channels. Then, another set of channel weights is generated by combining local 1D convolution to strengthen the interaction between channels and realize the dual weighted attention mechanism.

[0018] As a further implementation method, the weights generated by the SE mechanism are added to another set of channel weights generated, and further calibrated by a sigmoid function, and the final comprehensive weights are applied to the input feature map to achieve a weighted operation of the features.

[0019] As an optional implementation, the process of extracting and processing audio features from MFCC spectrogram features using the branch network architecture after adding the module includes: extracting MFCC features from the processed audio signal, and generating an MFCC feature map containing time series features of the audio signal by setting the sampling rate, Fourier transform size, frame skip size, number of MFCC coefficients, and number of Mel filters;

[0020] The generated MFCC feature map is input into a series of 1D convolutional layers, and the features are weighted by the SE-ECA fusion attention mechanism module after each convolution. The features processed by the SE-ECA fusion attention mechanism module are flattened, and the feature vector is further compressed through the fully connected layer to finally generate the MFCC feature embedding for classification.

[0021] As an optional implementation, the process of extracting and processing the audio feature of the Log-Mel spectrogram feature using the branch network architecture after adding the module includes: reshaping the input audio data and converting it through the Log-Mel spectrogram to generate the Log-Mel feature based on the audio signal;

[0022] Through the convolution layer, the input Log-Mel features are initially extracted, and then the Leaky ReLU activation function is applied to enhance the nonlinear ability. The SE_ECA_Block module is introduced, which integrates the SE module and the ECA mechanism to perform adaptive weight adjustment on the convolution output;

[0023] Use multiple convolution blocks, each of which applies the Leaky ReLU activation function, and introduce the SEECA Block module after feature extraction;

[0024] The spatial dimension of the feature map is downsampled through the maximum pooling layer, and channel conversion is performed on the pooled feature map through deeper convolution operations and residual connections;

[0025] The final convolutional features are max-pooled and flattened before entering a fully connected layer to generate an embedded representation of a fixed dimension.

[0026] As an optional implementation, the process of weighted averaging the global anomaly score and the local anomaly score includes adjusting the weights of the Gaussian mixture model and the local anomaly factor according to the actual application scenario. When the global anomaly is more than the set value, the Gaussian mixture model is given a higher weight; otherwise, the local anomaly factor is given a higher weight.

[0027] As an optional implementation, in the process of weighted averaging the global anomaly score and the local anomaly score, for each machine type and corresponding partial combination, the mean of the two embedding vectors is calculated, and the embedding vector of a normal training sample in the source domain is mixed with the embedding vector of a randomly selected sample in the target domain, so that each normal sample in the source domain generates a new mixed sample as its additional copy; these mixed samples are used together with the original samples in the source domain to train the Gaussian mixture model.

[0028] An unsupervised abnormal sound detection system for mechanical equipment based on domain generalization, comprising:

[0029] The sound data acquisition and preprocessing module is configured to acquire audio data emitted by the operation of different mechanical equipment, including normal audio data and abnormal audio data, and divide the collected audio data to obtain a training data set and a test data set;

[0030] The spectrogram feature extraction module is configured to extract MFCC spectrogram features and Log-Mel spectrogram features from the audio data samples in the training set respectively;

[0031] The audio feature extraction module is configured to construct a global context-based SE-ECA fusion attention mechanism module, respectively construct an MFCC branch network architecture and a Log-Mel branch network architecture, add the module to each branch network architecture, and respectively use the branch network architecture after adding the module to extract and process the audio features of the MFCC spectrogram features and the Log-Mel spectrogram features;

[0032] An audio classification module is configured to merge the audio feature embeddings extracted by the two branch network architectures in the channel dimension to form a comprehensive feature vector, classify the comprehensive feature vector, and stack the classification results with the true labels to form the final classification output;

[0033] A model building and training module is configured to build and train an anomaly detection model, wherein the anomaly detection model calculates a global anomaly score using a Gaussian mixture model, detects local anomalies using a local anomaly factor, and performs a weighted average of the global anomaly score and the local anomaly score;

[0034] The abnormal sound detection module is configured to evaluate the test data set using the trained anomaly detection model, calculate the log-likelihood value of each test sample, and use it as the final anomaly score.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] In view of the problems that the mechanical sounds of mechanical failures are difficult to obtain during the operation of mechanical equipment and the anomaly detection model has weak generalization ability, the present invention proposes an unsupervised abnormal sound detection method and system for mechanical equipment based on domain generalization, which has strong domain generalization ability.

[0037] The present invention provides a hybrid module that combines the SE (Squeeze and Excitation) mechanism with the ECA (Efficient Channel Attention) mechanism, which enhances the model's attention to channel-level information and improves the accuracy of classification tasks.

[0038] The present invention splices the embedding vectors of MFCC and Log-Mel Spectrogram, and the MFCC features are mainly used to describe the spectral characteristics of audio signals. The features extracted by Log-Mel Spectrogram provide richer information in the time-frequency domain, especially better details in the high-frequency part, can capture the time-frequency changes of the audio signal, and are suitable for describing complex audio signal characteristics; MFCC and Log-Mel Spectrogram describe the audio signal from different levels and angles respectively, and the model can simultaneously utilize low-level and high-level information to improve the description ability and classification accuracy of complex audio signals. Multi-feature fusion reduces the information redundancy problem caused by single feature representation and enhances the model's ability to capture key information in diversified audio signals. At the same time, the discriminant embedding is extracted based on a neural network trained by an AdaCos loss subcluster.

[0039] The present invention proposes a model based on the combination of global and local anomaly detection. The model first obtains the global anomaly score through a Gaussian mixture model (GMM), then applies a local anomaly factor (LOF) to perform fine-grained anomaly detection on the local area, and finally combines the scores of the two into a final anomaly score by weighted averaging, thereby improving the accuracy of anomaly detection. In order to improve the domain generalization ability, the present invention applies a domain mixing technology, that is, when estimating the embedding distribution with a GMM-LOF anomaly detection model, samples of the source domain and the target domain are mixed to obtain anomaly scores. No external data is used to increase data samples or train the system, and the generalization ability of the anomaly detection model is also enhanced.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0042] Figure 1 The present invention is a flowchart of a method for detecting abnormal sound of mechanical equipment unsupervisedly based on domain generalization according to an embodiment;

[0043] Figure 2 is a schematic diagram of a global context-based SE-ECA fusion attention mechanism module of an embodiment;

[0044] Figure 3 is a schematic diagram of the MFCC Net sub-network architecture of an embodiment;

[0045] Figure 4 The diagram is a schematic diagram of the Log-Mel Net sub-network architecture of an embodiment. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0049] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0050] Embodiment 1

[0051] An unsupervised abnormal sound detection method for mechanical equipment based on domain generalization, such as Figure 1 As shown, the following steps are included:

[0052] S1. Collect audio data emitted by different mechanical equipment, including normal audio data and abnormal audio data.

[0053] S2. Divide the collected audio data into two types of audio data sets: a training data set and a test data set.

[0054] S3. Extract MFCC spectrogram features and Log-Mel spectrogram features for the audio data samples in the training set respectively.

[0055] S4. Construct a global context-based SE-ECA fusion attention mechanism module. This module integrates and improves the SE and ECA attention mechanisms. This module obtains the global information of the feature map through global pooling and maximum pooling, and then combines local 1D convolution to strengthen the interaction between channels, thus realizing a double-weighted attention mechanism. This fusion not only enhances the model's ability to capture complex feature dependencies, but also strikes a balance between flexibility and parameter efficiency, capturing important features in audio data, thereby improving the classification effect of the model.

[0056] S5. For different feature maps, corresponding network branch sub-architectures are constructed: MFCC branch network architecture and Log-Mel branch network architecture, and the fused and improved global context-based SE-ECA fusion attention mechanism module is added to the MFCC network branch architecture and Log-Mel branch network architecture, and audio feature extraction and processing of the MFCC branch and the Log-Mel Spectrogram branch are performed.

[0057] S6. The features of the MFCC branch and the Log-Mel Spectrogram branch are embedded and merged in the channel dimension to form a comprehensive feature vector. The merged comprehensive feature vector is input into the SCAdaCos classification layer, which improves the classification accuracy by processing the subgroup features between classes. Finally, the classification results are stacked with the true labels through the Lambda layer to form the final classification output.

[0058] S7. Mixed training data and model training: During the training process, new training samples are generated through linear interpolation (Mixup technology). This method can effectively improve the generalization ability of the model and reduce the risk of overfitting. The combined features are input into the model for training. During the training process, the model continuously adjusts parameters through the optimizer to minimize the classification error.

[0059] S8, GMM-LOF anomaly detection: By first applying GMM to calculate the global anomaly score, we can ensure accurate modeling of the overall data distribution, and then use LOF to detect local anomalies to make up for the shortcomings of GMM in local patterns. The combination of the two can effectively improve the accuracy of anomaly detection, especially for complex scenarios where local anomalies exist in the data. Unlike directly using one of the two methods, the scores of the two are innovatively weighted averaged to avoid the limitations of a single method, and the contribution of GMM and LOF to the final result can be flexibly controlled by adjusting the weights, thereby enhancing the adaptability of the system. In multi-domain scenarios (such as different audio sources or different noise backgrounds), the generalization ability of the data is further improved through the domain mixup method. This method ensures that under multiple audio data sources, the model can balance the data characteristics of different fields and improve detection robustness.

[0060] S9. Use the trained anomaly model to evaluate the test dataset, calculate the log-likelihood value of each test sample, and use it as the final anomaly score. Compare the evaluation results with the true labels, and calculate the model's AUC (Area Under Curve) and pAUC (partial AUC) to measure the model's performance on different categories and domains.

[0061] S10. Generate corresponding anomaly scores and decision result files according to different test categories. Compare the model performance between the source domain and the target domain, and analyze the generalization ability of the model and the performance differences in different domains.

[0062] In this embodiment, for step S3, the specific operation of MFCC spectrogram feature extraction is: first, the original audio signal is subjected to short-time Fourier transform (STFT) to obtain a spectrogram. Then, the spectrogram is converted into an energy spectrogram of Mel frequency scale, and then the logarithm is taken, and finally the MFCC feature is obtained by discrete transform (DCT). The obtained MFCC feature is a two-dimensional matrix, and an appropriate dimensional transformation is performed before being input into the convolution layer in the network.

[0063] In this embodiment, for the extraction of the logarithmic Mel spectrogram, the original audio signal is first subjected to short-time Fourier transform to calculate the amplitude spectrum, which is then mapped to the Mel frequency scale and then subjected to logarithmic transformation to obtain the logarithmic Mel spectrogram. This spectrogram reflects the energy distribution of the signal at different frequencies, and the expressiveness of the low energy area is enhanced by logarithmic transformation.

[0064] In this embodiment, in step S4, in the traditional SE module, only global average pooling (GAP) is used to obtain the global information of the channel. In this embodiment, in order to enhance the focus on the salient areas in the feature map, the module introduces global maximum pooling (GMP). GMP can capture the strongest activation value in the feature map, so that the feature map pays more attention to the salient areas in the channel. The formula is as follows:

[0065]

[0066] Among them, H and W are the height and width of the feature map, and the average value of all elements in each channel is calculated to extract global features.

[0067]

[0068] Calculate the maximum value of all elements of each channel to extract the most significant global features.

[0069] In this embodiment, if Figure 2 As shown in the figure, the results of GAP and GMP are concatenated in the channel dimension, which makes the final feature consider the combination of the average information and the strongest information of the channel, thereby enriching the feature representation. After concatenation, it is expressed as:

[0070] context(x c )=[GAP(x c ),GMP(x c )](3)

[0071] The GAP and GMP results are concatenated in the channel dimension to form a context vector.

[0072] The concatenated feature S is processed twice by a fully connected layer (Dense layer), first by generating an intermediate representation through the ReLU activation function, and then by the sigmoid activation function to generate the weight of each channel. These weights will be used to recalibrate the channels of the input feature map.

[0073] z=ReLu(W1·context+b1)(4)

[0074] Among them, W1 represents the weight matrix, b1 represents the bias vector, and ReLu represents the activation function.

[0075] s=σ(W2·z+b2)(5)

[0076] Among them, W2 represents the weight matrix, b2 represents the bias vector, and σ represents the Sigmoid activation function.

[0077] In this embodiment, the input feature map is weighted by the SE mechanism to adjust the feature dimension between channels.

[0078]

[0079] Among them, s c represents the channel weight, x c Represents the input feature map.

[0080] The ECA module introduces a 1D convolution operation based on the channel weights generated by the SE module. The 1D convolution is applied to the features after global averaging through the Conv1D layer to generate another set of channel weights. This operation can comprehensively consider the global information within the channel and the local correlation between channels.

[0081]

[0082] Among them, Conv1D represents a one-dimensional convolution operation.

[0083] In this embodiment, the weight A generated by the SE module is added to the weight E generated by the ECA module and further calibrated by the sigmoid function. The final comprehensive weight is applied to the input feature map to achieve weighted operation of the features, thereby enhancing the model's attention to important channels.

[0084] w c =σ(s c +e c )(9)

[0085] Among them, s c represents SE weight, e c Represents the ECA weight.

[0086] The final output feature map.

[0087]

[0088] In this embodiment, in step S5, Figure 3 As shown, the specific steps for audio feature extraction and processing of the MFCC branch are:

[0089] First, the MFCC (Mel-Frequency Cepstral Coefficient) feature extraction is performed on the processed audio signal. By setting parameters such as sampling rate, Fourier transform size, frame skip size, number of MFCC coefficients, and number of Mel filters, an MFCC feature map containing the time series characteristics of the audio signal is generated.

[0090] Secondly, the generated MFCC feature maps are input into a series of 1D convolutional layers, and the features are weighted by fusing the improved SE-ECA module after each convolution. The SE-ECA module combines global average pooling and global maximum pooling, and uses the weights generated by 1D convolution to comprehensively consider the global information within the channel and the local correlation between channels to recalibrate the features.

[0091] Finally, the features processed by the SE-ECA module are flattened, and the feature vector is further compressed through a fully connected layer to finally generate the MFCC feature embedding for classification.

[0092] In this embodiment, if Figure 4 As shown in the figure, the specific steps of audio feature extraction and processing for the Log-Mel Spectrogram branch are:

[0093] First, Log-Mel spectrogram extraction: The input audio data is reshaped and converted through Log-Mel spectrogram to generate Log-Mel features based on the audio signal. To improve the robustness of feature representation, mean normalization and batch normalization operations are applied to the spectrogram.

[0094] Secondly, the first convolution module: a 7×7 convolution layer is used to perform preliminary feature extraction on the input Log-Mel features, and then a Leaky ReLU activation function is applied to enhance nonlinear capabilities. The SE_ECA_Block module is introduced, which combines the SE module and the ECA mechanism to further perform adaptive weight adjustment on the convolution output to improve the model's ability to perceive the importance of different channel features.

[0095] Subsequent convolutional modules: The following multiple convolutional blocks are stacked through multiple layers of 3×3 convolutional layers, using residual connections to maintain feature transferability. Each module applies the Leaky ReLU activation function and introduces the SE_ECA_Block module after feature extraction to further improve the adaptive weight allocation of feature channels.

[0096] Then perform pooling operation: In each module, the spatial dimension of the feature map is downsampled through the maximum pooling layer, gradually reducing the resolution of the feature map while retaining important global features. A 1×1 convolution is applied to the pooled feature map for channel conversion to maintain the consistency of feature dimensions.

[0097] Deep convolution module: As the network goes deeper, the number of channels in the convolution layer gradually increases (from 16 to 128), and the SE_ECA_Block module is used to further optimize the effect of feature extraction. Through deeper convolution operations and residual connections, the network can capture higher-dimensional abstract features.

[0098] Finally, embedding generation: The final convolutional features are processed by the maximum pooling operation and flattening, and then enter the fully connected layer to generate a fixed-dimensional embedding representation. After the batch normalization layer and the dense layer are processed, a 128-dimensional embedding vector of the audio is generated.

[0099] In this embodiment, in step S8, when training the GMM-LOF anomaly detection model, the advantages of global and local complement each other. GMM can capture global distribution well, while LOF is good at detecting local anomalies. Combining the two can avoid the failure problem of GMM in local anomaly detection, and at the same time make up for the shortcomings of LOF in capturing global anomalies. Especially in the cross-domain scenario of audio data, GMM provides a macroscopic global perspective, while LOF focuses on detail anomalies. By combining GMM and LOF, the system can not only identify the overall abnormal trend, but also effectively capture local small-amplitude anomalies, which is particularly important for anomaly detection in complex audio data.

[0100] Different from the traditional single anomaly detection method, in this embodiment, the weights of GMM and LOF can be adjusted according to the actual application scenario, so as to optimize the detection performance under different types of data. For example, when there are many global anomalies, GMM can be given a higher weight; and when local anomalies are significant, the contribution of LOF can be strengthened. At the same time, the technology of domain mixing is adopted.

[0101] Specifically, for each machine type and corresponding part combination, the mean of the two embedding vectors is first calculated, and the embedding vector of a normal training sample in the source domain is mixed with the embedding vector of a randomly selected sample in the target domain. In this way, each normal sample in the source domain generates a new mixed sample as its additional copy. Subsequently, these mixed samples are used to train the GMM together with the original samples in the source domain. This strategy effectively generates more samples that are correct in terms of machine type and part, but do not belong to the source or target domain alone. With these newly generated samples, the model can learn a more domain-independent embedding distribution, thereby improving the generalization ability on different domains.

[0102] In summary, this embodiment integrates and improves a hybrid module based on the SE (Squeeze and Excitation) mechanism and the ECA (Efficient Channel Attention) mechanism, named SE_ECA_Block, which can significantly enhance the model's attention to channel-level information and improve the accuracy in classification tasks. The improved dual attention mechanism of SE and ECA enables the model to effectively model and weight channel information at both the global and local levels. Through this dual mechanism, the model can more accurately identify and distinguish features of different categories in classification tasks, thereby improving the overall classification accuracy.

[0103] This embodiment improves classification accuracy by fusing MFCC and Log-Mel Spectrogram embedding vectors. MFCC is a commonly used feature extraction method in audio signal processing, which can effectively capture low-level information of audio, such as fundamental frequency and resonance peak. MFCC features are mainly used to describe the spectral characteristics of audio signals. The features extracted by Log-Mel Spectrogram provide richer information in the time-frequency domain, especially better details in the high-frequency part. It can capture the time-frequency changes of audio signals and is suitable for describing complex audio signal features. MFCC and Log-Mel Spectrogram describe audio signals from different levels and angles respectively. By merging the embedding vectors of these two features, the model can simultaneously utilize low-level and high-level information, thereby improving the description ability and classification accuracy of complex audio signals. Through the fusion of multiple features, the model can effectively reduce the information redundancy problem caused by single feature representation. The complementarity between different features enables the model to capture key information more accurately and improve overall performance when facing diverse audio signals.

[0104] This embodiment constructs and trains a model based on the combination of global and local anomaly detection. The model first obtains the global anomaly score through a Gaussian mixture model (GMM), then applies a local anomaly factor (LOF) to perform fine-grained anomaly detection on the local area, and finally combines the scores of the two into the final anomaly score by weighted averaging.

[0105] Embodiment 2

[0106] An unsupervised abnormal sound detection system for mechanical equipment based on domain generalization, comprising:

[0107] The sound data acquisition and preprocessing module is configured to acquire audio data emitted by the operation of different mechanical equipment, including normal audio data and abnormal audio data, and divide the collected audio data to obtain a training data set and a test data set;

[0108] The spectrogram feature extraction module is configured to extract MFCC spectrogram features and Log-Mel spectrogram features from the audio data samples in the training set respectively;

[0109] The audio feature extraction module is configured to construct a global context-based SE-ECA fusion attention mechanism module, respectively construct an MFCC branch network architecture and a Log-Mel branch network architecture, add the module to each branch network architecture, and respectively use the branch network architecture after adding the module to extract and process the audio features of the MFCC spectrogram features and the Log-Mel spectrogram features;

[0110] An audio classification module is configured to merge the audio feature embeddings extracted by the two branch network architectures in the channel dimension to form a comprehensive feature vector, classify the comprehensive feature vector, and stack the classification results with the true labels to form the final classification output;

[0111] A model building and training module is configured to build and train an anomaly detection model, wherein the anomaly detection model calculates a global anomaly score using a Gaussian mixture model, detects local anomalies using a local anomaly factor, and performs a weighted average of the global anomaly score and the local anomaly score;

[0112] The abnormal sound detection module is configured to evaluate the test data set using the trained anomaly detection model, calculate the log-likelihood value of each test sample, and use it as the final anomaly score.

[0113] It can be understood that the above-mentioned units / modules can be separately or completely combined into one or several other units / modules, or one (some) of the units can be further divided into multiple functionally smaller units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.

[0114] The above modules of the system are divided based on logical functions. In practical applications, the function of one module can also be implemented by multiple modules, or the functions of multiple modules can be implemented by one module. For example, the sound data acquisition and preprocessing module of this embodiment may include:

[0115] A sound data acquisition module is configured to acquire audio data emitted by the operation of different mechanical equipment, including normal audio data and abnormal audio data;

[0116] The sound data division module is configured to divide the collected audio data into a training data set and a test data set.

[0117] Similarly, in other embodiments of the present application, the system may also include other units / modules. In actual applications, these functions may also be implemented with the assistance of other units, and may be implemented by collaboration of multiple units.

[0118] According to another embodiment of the present application, the system described in this embodiment can be constructed and the method of Embodiment 1 can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding method described in Embodiment 1 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.

Claims

1. A method for detecting abnormal sound of mechanical equipment based on domain generalization, characterized in that: The following steps are involved: Acquire audio data emitted by different mechanical equipment, including normal audio data and abnormal audio data, divide the collected audio data into training data sets and test data sets; Extract MFCC spectrogram features and Log-Mel spectrogram features from the audio data samples in the training set; Constructing a global context-based SE-ECA fusion attention mechanism module, respectively constructing an MFCC branch network architecture and a Log-Mel branch network architecture, adding the module to each branch network architecture, and respectively using the branch network architecture after adding the module to extract and process audio features of the MFCC spectrogram features and the Log-Mel spectrogram features; The audio feature embeddings extracted by the two branch network architectures are merged in the channel dimension to form a comprehensive feature vector, the comprehensive feature vector is classified, and the classification results are stacked with the true labels to form the final classification output; Constructing and training an anomaly detection model, wherein the anomaly detection model uses a Gaussian mixture model to calculate a global anomaly score, uses a local anomaly factor to detect local anomalies, and performs a weighted average of the global anomaly score and the local anomaly score; The trained anomaly detection model is used to evaluate the test dataset and the log-likelihood value of each test sample is calculated and used as the final anomaly score.

2. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The process of extracting MFCC spectrogram features from the audio data samples in the training set includes performing short-time Fourier transform on the original audio signal to obtain a spectrogram, converting the spectrogram into an energy spectrum on a Mel frequency scale, taking its logarithm, and finally obtaining the MFCC features through discrete transformation.

3. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The process of extracting Log-Mel spectrogram features from the audio data samples in the training set includes performing a short-time Fourier transform on the original audio signal to calculate the amplitude spectrum, mapping it to the Mel frequency scale, and then performing a logarithmic transform to obtain a log-Mel spectrogram.

4. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The global context-based SE-ECA fusion attention mechanism module combines the SE and ECA attention mechanisms, obtains the global information of the feature map through global pooling and maximum pooling, and concatenates the global information obtained by global pooling and maximum pooling in the channel dimension to form a context vector; The concatenated features are processed by two fully connected layers to obtain the input feature map, which is weighted by the SE mechanism to adjust the feature dimensions between channels. Then, another set of channel weights is generated by combining local 1D convolution to strengthen the interaction between channels and realize the dual weighted attention mechanism.

5. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 4, characterized in that: The weights generated by the SE mechanism are added to another set of channel weights generated, and further calibrated by the sigmoid function, and the final comprehensive weights are applied to the input feature map to achieve weighted operation of the features.

6. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The process of extracting and processing audio features from MFCC spectrogram features using the branch network architecture after adding the module includes: extracting MFCC features from the processed audio signal, and generating an MFCC feature map containing time series features of the audio signal by setting the sampling rate, Fourier transform size, frame skip size, number of MFCC coefficients, and number of Mel filters; The generated MFCC feature map is input into a series of 1D convolutional layers, and the features are weighted by the SE-ECA fusion attention mechanism module after each convolution. The features processed by the SE-ECA fusion attention mechanism module are flattened, and the feature vector is further compressed through the fully connected layer to finally generate the MFCC feature embedding for classification.

7. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The process of extracting and processing audio features from Log-Mel spectrogram features using the branch network architecture after adding the module includes: reshaping the input audio data and converting it through Log-Mel spectrogram to generate Log-Mel features based on the audio signal; Through the convolution layer, the input Log-Mel features are initially extracted, and then the Leaky ReLU activation function is applied to enhance the nonlinear ability. The SE_ECA_Block module is introduced, which integrates the SE module and the ECA mechanism to perform adaptive weight adjustment on the convolution output; Using multiple convolution blocks, each convolution block applies the Leaky ReLU activation function, and introduces the SE_ECA_Block module after feature extraction; The spatial dimension of the feature map is downsampled through the maximum pooling layer, and channel conversion is performed on the pooled feature map through deeper convolution operations and residual connections; The final convolutional features are max-pooled and flattened before entering a fully connected layer to generate an embedded representation of a fixed dimension.

8. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: The process of weighted averaging the global anomaly score and the local anomaly score includes adjusting the weights of the Gaussian mixture model and the local anomaly factor according to the actual application scenario. When the global anomaly is greater than the set value, the Gaussian mixture model is given a higher weight; otherwise, the local anomaly factor is given a higher weight.

9. The method for detecting abnormal sound of mechanical equipment based on domain generalization according to claim 1, characterized in that: In the process of weighted averaging the global anomaly score and the local anomaly score, for each machine type and corresponding part combination, the mean of the two embedding vectors is calculated, and the embedding vector of a normal training sample in the source domain is mixed with the embedding vector of a randomly selected sample in the target domain, so that each normal sample in the source domain generates a new mixed sample as its additional copy; these mixed samples are used together with the original samples in the source domain to train the Gaussian mixture model.

10. An unsupervised abnormal sound detection system for mechanical equipment based on domain generalization, characterized in that: include: The sound data acquisition and preprocessing module is configured to acquire audio data emitted by the operation of different mechanical equipment, including normal audio data and abnormal audio data, and divide the collected audio data to obtain a training data set and a test data set; The spectrogram feature extraction module is configured to extract MFCC spectrogram features and Log-Mel spectrogram features from the audio data samples in the training set respectively; The audio feature extraction module is configured to construct a global context-based SE-ECA fusion attention mechanism module, respectively construct an MFCC branch network architecture and a Log-Mel branch network architecture, add the module to each branch network architecture, and respectively use the branch network architecture after adding the module to extract and process the audio features of the MFCC spectrogram features and the Log-Mel spectrogram features; An audio classification module is configured to merge the audio feature embeddings extracted by the two branch network architectures in the channel dimension to form a comprehensive feature vector, classify the comprehensive feature vector, and stack the classification results with the true labels to form the final classification output; A model building and training module is configured to build and train an anomaly detection model, wherein the anomaly detection model calculates a global anomaly score using a Gaussian mixture model, detects local anomalies using a local anomaly factor, and performs a weighted average of the global anomaly score and the local anomaly score; The abnormal sound detection module is configured to evaluate the test data set using the trained anomaly detection model, calculate the log-likelihood value of each test sample, and use it as the final anomaly score.

Images