Industrial equipment abnormal sound signal detection method and system based on twin attention adversarial network

Abstract

The application discloses an industrial equipment abnormal sound signal detection method and system based on a twin attention adversarial network. The method comprises the following steps: performing logarithmic mel energy feature extraction on an original sound signal; constructing a twin network with a shared automatic encoder architecture; sharing the weights of the twin encoder to extract the feature representation of the logarithmic mel energy of the source domain and the target domain; constructing a domain discriminator to align the feature representations of the source domain and the target domain to learn the invariant feature representation thereof; implementing good reconstruction of the source domain and the target domain by the twin decoder; constructing an overall loss function to train the twin attention adversarial network by using back propagation and Adam optimization; inputting the training set and the verification set into the twin attention adversarial network for training, calculating an abnormal score through a reconstruction error, and then obtaining a threshold value, and outputting the state of the industrial equipment according to the threshold value; inputting the test set into the trained twin attention adversarial network, calculating an abnormal score, determining a threshold value, and determining the state of the industrial equipment according to the threshold value.

Description

Technical Field

[0001] This invention belongs to the field of abnormal acoustic signal detection technology under domain drift scenarios, specifically relating to a method and system for detecting abnormal acoustic signals in industrial equipment based on twin attention adversarial networks. Background Technology

[0002] With the advent of the era of intelligent manufacturing, driven by supporting technologies such as artificial intelligence algorithms, cyber-physical systems, cloud computing, big data, and the Internet of Things, modern industrial production processes (such as aerospace, equipment manufacturing, and textiles) are developing towards intelligence, efficiency, and integration. However, the composition and structure of modern industrial equipment are becoming increasingly complex, and the connections between various components are becoming increasingly close. If an abnormality occurs in a component of the equipment, it may directly affect product production, leading to unnecessary economic losses or even casualties. Therefore, it is essential to adopt certain anomaly detection technologies. Among the many anomaly detection signal acquisition methods, acoustic signals have advantages such as simple equipment installation, low cost, and non-contact measurement. Consequently, the detection of abnormal acoustic signals in industrial equipment has attracted widespread attention from researchers. Abnormal sound detection (ASD) refers to identifying whether the sounds emitted by machines are normal or abnormal. Abnormal sounds may be caused by abnormal events such as wear, machine malfunction, or improper operation. Abnormal sound signal detection not only ensures the safe and stable performance of industrial production processes but also improves industrial production efficiency.

[0003] Since abnormal acoustic signal data are rarely available in actual industrial production, unsupervised abnormal acoustic signal detection methods are usually employed. However, it's not just the abnormal state of acoustic signals from industrial equipment that presents challenges. When the operating conditions and environment of industrial equipment change (i.e., differences exist in operating speed, machine load, ambient noise, viscosity, etc.), the data distribution in the source domain (normal scenario) and the target domain (drift scenario) will also change, which is very challenging for abnormal acoustic signal detection.

[0004] Many methods have been proposed in this field to reduce the distributional differences between the source and target domains. Drift learning, a recent breakthrough, can handle distributional differences between samples, but it suffers from negative transfer effects. GANs primarily fit the probability distribution through adversarial training mechanisms between the two domains, but their training requires reaching a "Nash equilibrium." Furthermore, the unique structure of Siamese networks can handle feature representations well under domain drift conditions, further improving the model's representational power. Based on this, Siamese adversarial networks have emerged in the field of computer vision, typically training domain classifiers through game theory to distinguish data extracted from the source and target domains. Therefore, applying Siamese adversarial networks to anomalous acoustic signal detection in domain drift scenarios can better meet the needs of practical industrial applications.

[0005] However, different frequency domain features of acoustic signals from various industrial equipment have different effects, so it is necessary to select useful spectral features and filter out useless information. If the extracted frequency domain features are directly input into the model for training, although some detection performance can be achieved, it may limit the model's detection performance and reduce its robustness. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, this invention proposes a method and system for detecting abnormal acoustic signals in industrial equipment based on a Siamese attention adversarial network. The invention first extracts the log-Mel energy features of normal acoustic signals in the source and target domains. Then, the extracted features from the two domains are fed into two identical encoders with frequency-domain-based attention mechanisms. Next, a domain discriminator is constructed to represent the features of the source and target domains, and a Siamese decoder achieves good reconstruction of the source and target domains. Then, an overall loss function is constructed, and the Siamese attention adversarial network is trained using backpropagation and Adam optimization. Finally, an anomaly score is calculated based on the reconstruction error to obtain a threshold, and the state of the industrial equipment is output according to the threshold.

[0007] The technical solution adopted by this invention to solve its technical problem uses a Siamese attention adversarial network model, as detailed below:

[0008] A method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks includes the following steps:

[0009] (I) Model Training Phase

[0010] Step 1-1: Extract log-Mel energy features from the original acoustic signals in the source and target domains; preferably, this includes pre-emphasis, framing, windowing, fast Fourier transform, Mel filter bank, and logarithmic transform to obtain their log-Mel energy features. .

[0011] Steps 1-2: Construct a twin network with a shared autoencoder architecture, consisting of an encoder and a decoder.

[0012] Steps 1-3: The twin encoders share weights and include a frequency-domain-based attention mechanism for better extraction of feature representations of the source and target domain log-Mel energy.

[0013] Steps 1-4: Construct a domain discriminator, align the feature representations of the source and target domains, and learn their invariant feature representations.

[0014] Steps 1-5: The twin decoder achieves good reconstruction of the source and target domains. The calculation formula is as follows:

[0015]

[0016] in , These are the log-Mel ability characteristics of the source and target domains, respectively. , They are , The reconstruction These are the learnable weight parameters of the decoder. , These are the number of input log-Mel energy features in the source and target domains, respectively.

[0017] Steps 1-6: Construct the overall loss function, and use backpropagation and Adam optimization to train the Siamese attention adversarial network. The calculation formula is as follows:

[0018]

[0019] in , These are the log-Mel ability characteristics of the source and target domains, respectively. , and Let represent the learnable weight parameters of the discriminator D, the encoder GEF, and the decoder GD, respectively. , These are the standard cross-entropy loss functions for the encoder and decoder, respectively, L. r The reconstruction loss function.

[0020] Steps 1-7: Prepare the training set X train and validation set X val The input is fed into a trained Siamese attention adversarial network, and anomaly scores are calculated by reconstructing errors to obtain a threshold. Finally, the state of the industrial equipment is output based on the threshold.

[0021] (II) Anomaly Detection Phase

[0022] For a collected unknown acoustic signal data, the following processing is performed to determine whether it is abnormal:

[0023] Step 2-1: Extract the log-Mel energy features according to Step 1-1 in (I). .

[0024] Step 2-2: Extract the log-Mel energy features from Step 2-1 Anomaly scores were obtained when the input was fed into a Siamese attention adversarial network. .

[0025] Steps 2-3: Determine the signal based on a threshold (H). Signals above the threshold are considered abnormal, while signals below the threshold are considered normal.

[0026] .

[0027] This invention can effectively detect abnormal acoustic signals when the data distributions in the source and target domains are inconsistent, and it has good generalization ability for domain drift scenarios.

[0028] Preferably, step 1-1 is as follows:

[0029] Step 1-1: For pre-weighting, by calculate, For input signal, This is the output after pre-emphasis. The pre-weighting factor is set to 0.97;

[0030] Steps 1-2: For frames, the frame length is... Frame shift is ;

[0031] Steps 1-3: For windowing, select a Hamming window ω(n), and calculate it using the formula x(n) = ω(n)s(n), where s(n) is the width of the window and is the nth sampling point of N;

[0032] Steps 1-4: For Fast Fourier Transform, the signal obtained from the above steps can be converted to the frequency domain. The calculation formula is as follows: ,in The input sound signal is j, where j is the imaginary unit. for The corresponding frequency domain signal, This is the number of points in the Fourier transform, usually taken as 512;

[0033] Steps 1-5: For the Mel filter bank, the main task is to weight the frequency domain information obtained in steps 1-4 to obtain the energy of each Mel frequency band, in order to remove noise and extract the useful signal. Assuming there are M triangular filters in total, the frequency response H of the m-th Mel filter is... m (k) is calculated using the formula, where f(m) is the center frequency:

[0034]

[0035] Step 1.6: For logarithmic operations, take the logarithm of the Mel frequency band energy obtained in Steps 1-5 to obtain the logarithmic Mel energy of each frame signal after filtering by the m-th Mel filter. The calculation formula is:

[0036]

[0037] Preferably, steps 1-3 are implemented as follows:

[0038] Steps 1-3.1: The working principle of the frequency domain-based attention mechanism is roughly as follows: First, the input X is passed through a fully connected layer with n hidden units, and then the spectral attention weights of the input data are calculated using a sigmoid activation function. Next, the weights are normalized to obtain the normalized spectral attention weights W. n,t Finally, W n,t Multiply by the input X to obtain the weighted features The calculation formulas for the relevant processes are as follows:

[0039]

[0040] Where X represents the spectral characteristics of the input acoustic signal, and V n and b n Let represent the weights and biases of the nth hidden unit in the fully connected layer, respectively. It is the number of Mel bins. Represents element-wise multiplication;

[0041] Step 1-3.2: Extract the feature representations of the log-Mel energy of the source and target domains using GEF(x) respectively. s ) and GEF(x t ) is obtained, where GEF(.) represents the twin encoder.

[0042] Preferably, steps 1-4 are implemented as follows:

[0043] Step 1-4.1: Construct a domain discriminator and attempt to distinguish source and target features from the outputs of the two GEFs. The calculation formula is the standard binary cross-entropy loss, and the relevant calculations are as follows:

[0044]

[0045] in , These are the log-Mel capability features of the source and target domains, respectively, where D and GEF represent the discriminator and encoder, respectively, and W... D and W GEF These represent the learnable weight parameters for D and GEF, respectively.

[0046] Step 1-4.2: Align the feature representations of the source and target domains. The calculation formula is the maximum mean difference, which aims to match the data distributions of the source and target domains. Here, ||.|2 represents L2 regularization.

[0047] .

[0048] Preferably, steps 1-7 are as follows:

[0049] Step 1-7.1: Process the training set according to Step 1-1 to obtain the log-Mel energy features of the source and target domains corresponding to the training set;

[0050] Step 1-7.2: Input the features obtained in Step 1-7.1 into the Siamese Attention Adversarial Network (SAD). The SAD consists of GEF, GD, and D. GEF consists of a frequency-domain-based attention mechanism module and six dense layers, each followed by batch normalization and ReLU activation. GD consists of six dense layers; notably, the last dense layer outputs the prediction directly, as no classification transformation is needed. D consists of two dense layers, each followed by batch normalization, ELU activation, and Dropout regularization.

[0051] Steps 1-7.3: Optimize according to steps 1-6, and train the Siamese attention adversarial network. The anomaly score is the average of the sum of the absolute values ​​of the differences between normal samples and anomaly samples. The formula for calculating the anomaly score is as follows:

[0052]

[0053] Where, x k The log-Mel energy characteristic at time k is... For the reconstructed signal features, K is the number of time frames.

[0054] F and P represent the number of Mel filter banks and the number of context windows, respectively, and ||.|2 represents L2 regularization;

[0055] Step 1-7.4: Determine the state of the industrial equipment based on the threshold (H). The threshold is defined as follows, assuming... obey distributed, The parameters of the distribution are determined by Histogram estimation is given. The anomaly score detection threshold is determined as follows. The 90th percentile of the distribution is used to determine whether an abnormal score of a sound signal segment is greater than this threshold; otherwise, it is considered normal.

[0056] This invention also discloses an industrial equipment abnormal acoustic signal detection system based on a twin attention adversarial network, which specifically includes the following modules:

[0057] Model training module: Extracts log-Mel energy features from the original acoustic signal; constructs a Siamese network with a shared autoencoder architecture; employs shared weights in the Siamese encoder, including a frequency-domain-based attention mechanism, to extract feature representations of the log-Mel energy in the source and target domains; constructs a domain discriminator to align the feature representations of the source and target domains to learn their invariant feature representations; a Siamese decoder reconstructs the source and target domains; constructs an overall loss function and uses backpropagation and Adam optimization to train the Siamese attention adversarial network; inputs the training and validation sets into the trained Siamese attention adversarial network, calculates anomaly scores based on reconstruction errors to obtain a threshold, and outputs the state of the industrial equipment based on the threshold.

[0058] Anomaly detection module: For the acquired unknown acoustic signal data, the following processing is performed to determine whether it is abnormal: Extract log-Mel energy features. The extracted log-Mel energy features Anomaly scores were obtained when the input was fed into a Siamese attention adversarial network. The sound signal is considered abnormal if it is greater than the threshold H, and normal if it is less than the threshold H.

[0059] The technical terms used in this invention are explained as follows:

[0060] The Fourier transform is a digital signal processing technique that converts a time-domain signal into the frequency domain for representation and analysis. Through the Fourier transform, the frequency composition of the signal, as well as the amplitude and phase information of each frequency component, can be obtained. Notably, the Fast Fourier Transform (FFT) algorithm optimizes the computation process by utilizing the periodicity and symmetry of the signal, significantly reducing computation time.

[0061] Log-Mel-band energy is a commonly used feature extraction method in the field of acoustic signals. It is the logarithmic energy obtained directly from the energy of a filter bank. Because it can effectively capture the frequency information in audio signals and has high robustness and stability, it can more accurately identify abnormal sound signals.

[0062] Frequency-based attention (FAM) is a type of attention mechanism that learns the importance of each frequency component, avoiding the information loss problem caused by directly using the spectrum of the entire sequence, thus generating feature representations that are more conducive to network training. Therefore, frequency-based attention mechanisms are widely used in fields such as natural language processing, speech recognition, and computer vision.

[0063] The beneficial effects of this invention are as follows:

[0064] For practical industrial applications, especially those where inconsistent data distributions between the source and target domains arise due to changes in operating conditions and environment, this invention reduces the data distribution differences between the source and target domains through a Siamese attention adversarial network, effectively learning invariant feature representations of the source and target domains. This invention is highly adaptable, better suited to various industrial equipment (such as valves, pumps, fans, slide rails, gearboxes, etc.) in real-world industrial scenarios. It also boasts high stability, with the robustness and stability of the model gradually enhanced through continuous learning and adaptation via Siamese attention adversarial networks. Furthermore, this invention exhibits strong scalability, adapting to different data distributions and task requirements, demonstrating high scalability and flexibility.

[0065] Compared to directly inputting the extracted log-Mel energy features into the model for training, this invention combines a frequency domain-based attention mechanism to effectively extract the required log-Mel energy frequency components of the acoustic signal. This not only enhances feature representation and improves detection performance, but also reduces model complexity and accelerates model training, offering advantages for practical industrial applications. Attached Figure Description

[0066] Figure 1 This is a flowchart of a preferred embodiment of the present invention for detecting abnormal acoustic signals in industrial equipment based on a twin attention adversarial network.

[0067] Figure 2 This is a diagram of the log-Mel energy extraction process (Mpro) according to a preferred embodiment of the present invention;

[0068] Figure 3 This is a structural diagram of a twin attention adversarial network according to a preferred embodiment of the present invention;

[0069] Figure 4 This is a flowchart of a preferred embodiment of the present invention for detecting abnormal acoustic signals in industrial equipment based on a twin attention adversarial network;

[0070] Figure 5 This is a diagram of industrial equipment according to a preferred embodiment of the present invention;

[0071] Figure 6 Block diagram of an industrial equipment abnormal acoustic signal detection system based on a twin attention adversarial network, according to a preferred embodiment of the present invention. Detailed Implementation

[0072] The following uses the publicly available industrial equipment dataset MIMII DUE as an example to further illustrate the present invention using an abnormal acoustic signal detection method based on Siamese attention adversarial networks. The following description is for illustrative purposes only and does not impose any limitations on the present invention.

[0073] In this embodiment, the MIMII DUE sound dataset for investigating and inspecting faulty industrial machines is used. It includes industrial equipment types such as valve, pump, fan, slider, and gearbox. Each type of industrial equipment consists of three parts (00, 01, and 02). For each part, the training set provides approximately 1000 normal sound clips from the source domain and three normal sound clips from the target domain. The test set provides approximately 100 normal and abnormal sound clips from both the source and target domains. The sampling rate is 16 kHz, the duration is 10 s, and the sampling encoding is 16-bit signed integer PCM. Furthermore, each sound signal clip is mixed with background noise recorded from multiple real factories and machine sounds simulating real-world environments, including pollution, leaks, rotational imbalances, and rail damage.

[0074] like Figure 1-5 As shown in the figure, this embodiment of a method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks specifically includes:

[0075] I. Model Training Phase:

[0076] Step 1: Extract log-Mel energy features from the original acoustic signal data in the source and target domains, including pre-emphasis, framing, windowing, fast Fourier transform, and log-Mel filtering, to obtain their log-Mel energy features. The specific steps are as follows:

[0077] Step 1.1: For pre-weighting, the calculation formula is as follows: , For input signal, The pre-weighting factor is set to 0.97;

[0078] Step 1.2: For frames, the frame length is... Frame shift is , Take 64 milliseconds;

[0079] Step 1.3: For windowing, select a Hamming window ω(n), and calculate it using the formula x(n) = ω(n)s(n), where s(n) is the nth sampling point of the window with a width of N;

[0080] Step 1.4: For the Fast Fourier Transform, the calculation formula is as follows: ,in For input sound signals, for The corresponding frequency domain signal, The number of points for the Fourier transform is 512.

[0081] Step 1.5: For the Mel filter bank, the main step is to weight the frequency domain information obtained in Step 1.4 to obtain the energy of each Mel frequency band, i.e., H. m (k), its calculation formula is:

[0082]

[0083] Where f(m) is the center frequency;

[0084] Step 1.6: For logarithmic operations, take the logarithm of the Mel frequency band energy obtained in Step 1.5 to obtain the logarithmic Mel energy of each frame signal after filtering by the m-th Mel filter. The calculation formula is:

[0085]

[0086] Step 2: Construct a twin network with a shared autoencoder architecture, consisting of an encoder and a decoder.

[0087] Step 3: The twin encoders share weights and include a frequency-domain-based attention mechanism to better extract feature representations of the log-Mel energy of the source and target domains. The specific implementation of this step is as follows:

[0088] Step 3.1: The input log-Mel energy features are weighted using a frequency-domain-based attention mechanism, calculated as follows:

[0089]

[0090] Where X represents the spectral characteristics of the input acoustic signal, and V n and b n Let represent the weights and biases of the nth hidden unit in the fully connected layer, respectively. It is the number of Mel bins. Represents element-wise multiplication;

[0091] Step 3.2: Extract the feature representations of the log-Mel energy of the source and target domains using GEF(x) respectively. s ) and GEF(x t ) is obtained, where GEF(.) represents the twin encoder.

[0092] Step 4: Construct a domain discriminator, aligning the feature representations of the source and target domains to learn their invariant feature representations; this step is detailed below:

[0093] Step 4.1: Construct a domain discriminator and attempt to distinguish source and target features from the outputs of the two GEFs. The calculation formula is the standard binary cross-entropy loss, as follows:

[0094]

[0095] Step 4.2: Align the feature representations of the source and target domains. The calculation formula is the maximum mean difference, which aims to match the data distributions of the source and target domains. The calculation formula is as follows:

[0096]

[0097] Step 5: The twin decoder achieves good reconstruction of the source and target domains. The calculation formula is as follows:

[0098]

[0099] Step 6. Construct the overall loss function, and use backpropagation and Adam optimization to train the Siamese attention adversarial network. The relevant calculation formula is as follows:

[0100]

[0101] Step 7. Transfer the training set X train and validation set X val The input is fed into a trained Siamese attention adversarial network, where anomaly scores are calculated based on reconstruction errors to obtain a threshold. Finally, the state of the industrial equipment is output based on the threshold. The specific steps are as follows:

[0102] Step 7.1: Process the training set according to Step 1 to obtain the log-Mel energy features of the source and target domains corresponding to the training set;

[0103] Step 7.2: Input the features obtained in Step 7.1 into the Siamese Attention Adversarial Network (SADN). The SADN consists of GEF, GD, and D. GEF receives a 640-dimensional feature vector and maps it to 16-dimensional features. This network consists of a frequency-domain-based attention mechanism module and six dense layers. Each dense layer is followed by batch normalization and a ReLU activation function. GD receives a 16-dimensional feature vector and maps it to 640-dimensional features. This network consists of six dense layers. Notably, the last dense layer directly outputs the prediction result, as direct output of the prediction result eliminates the need for classification transformation. D consists of two dense layers. Each dense layer is followed by batch normalization, ELU activation, and Dropout regularization, with Dropout=0.5.

[0104] training set The input is a stacked autoencoder for training. The SAE consists of 5 identical AEs. The encoder of a single AE consists of 5 fully connected layers with dimensions of 512-512-512-512-32. The decoder is the opposite, with dimensions of 32-512-512-512-512. The Adam optimizer is used with a learning rate of 0.001, 100 epochs, and a batch size of 512. When the validation loss stops improving within 10 epochs, the best model is saved and the training process is stopped early.

[0105] Step 7.3: Optimize according to Step 6, and train the Siamese attention adversarial network. The anomaly score is the average of the sum of the absolute values ​​of the differences between normal samples and anomaly samples. The formula for calculating the anomaly score is as follows:

[0106]

[0107] Where, x k The log-Mel energy characteristic at time k is... For the reconstructed signal features, K is the number of time frames.

[0108] F and P represent the number of Mel filter banks and the number of context windows, respectively, and ||.|2 represents L2 regularization;

[0109] Step 7.4: Determine the state of the industrial equipment based on the threshold (H). The threshold is defined as follows, assuming... obey distributed, The parameters of the distribution are determined by Histogram estimation is given. The anomaly score detection threshold is determined as follows. The 90th percentile of the distribution is used to determine whether an abnormal score of a sound signal segment is greater than this threshold; otherwise, it is considered normal.

[0110] II. Anomaly Detection Phase:

[0111] Step 1: Extract the log-Mel energy features from the corresponding source and target domains in the same manner as during the model training phase. .

[0112] Step 2: Extract the log-Mel energy features from Step 1. Anomaly scores were obtained when the input was fed into a Siamese attention adversarial network. .

[0113] Step 3: Determine the signal based on a threshold (H). Signals above the threshold are considered abnormal, while signals below the threshold are considered normal. This is represented as follows:

[0114] .

[0115] like Figure 6 As shown, this embodiment discloses an industrial equipment abnormal acoustic signal detection system based on a Siamese attention adversarial network. Based on the above method embodiment, it specifically includes the following modules:

[0116] Model training module: Extracts log-Mel energy features from the original acoustic signal; constructs a Siamese network with a shared autoencoder architecture; employs shared weights in the Siamese encoder, including a frequency-domain-based attention mechanism, to extract feature representations of the log-Mel energy in the source and target domains; constructs a domain discriminator to align the feature representations of the source and target domains to learn their invariant feature representations; a Siamese decoder reconstructs the source and target domains; constructs an overall loss function and uses backpropagation and Adam optimization to train the Siamese attention adversarial network; inputs the training and validation sets into the trained Siamese attention adversarial network, calculates anomaly scores based on reconstruction errors to obtain a threshold, and outputs the state of the industrial equipment based on the threshold.

[0117] Anomaly detection module: For the acquired unknown acoustic signal data, the following processing is performed to determine whether it is abnormal: Extract log-Mel energy features. The extracted log-Mel energy features Anomaly scores were obtained when the input was fed into a Siamese attention adversarial network. The sound signal is considered abnormal if it is greater than the threshold H, and normal if it is less than the threshold H.

[0118] Other aspects of this embodiment can be found in the above method embodiments.

[0119] Any aspects not covered in this invention are applicable to existing technologies.

[0120] The specific embodiments of the present invention have been described above to enable those skilled in the art to understand the present invention. However, the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

Claims

1. A method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks, characterized in that... Specifically, the steps include the following: (I) Model Training Phase Step 1-1: Extract log-Mel energy features from the original acoustic signal; Steps 1-2: Construct a twin network with a shared autoencoder architecture; Steps 1-3: Use a twin encoder to share weights, including a frequency domain-based attention mechanism, to extract feature representations of the log-Mel energy of the source and target domains; Steps 1-4: Construct a domain discriminator, align the feature representations of the source and target domains, and learn their invariant feature representations; Steps 1-5: The twin decoder reconstructs the source and target domains using the following calculation formula: in , These are the log-Mel energy characteristics of the source and target domains, respectively. , They are , The reconstruction features, These are the learnable weight parameters of the decoder. , These are the number of input log-Mel energy features in the source and target domains, respectively. Steps 1-6: Construct the overall loss function, and use backpropagation and Adam optimization to train the Siamese attention adversarial network. The calculation formula is as follows: in , These are the log-Mel energy characteristics of the source and target domains, respectively. , and Let D, GEF, and GD represent the learnable weight parameters of the discriminator, encoder, and decoder, respectively. , These are the standard cross-entropy loss functions for the encoder and decoder, respectively, L. r For reconstruction loss function; Steps 1-7: Prepare the training set X train and validation set X val The input is fed into a trained Siamese attention adversarial network, and anomaly scores are calculated by reconstructing errors to obtain a threshold. The state of the industrial equipment is then output based on the threshold. (II) Anomaly Detection Phase For the collected unknown acoustic signal data, the following processing is performed to determine whether it is abnormal: Step 2-1: Extract the log-Mel energy features according to Step 1-1 in the model training phase (I). ; Step 2-2: Extract the log-Mel energy features Anomaly scores were obtained when the input was fed into a Siamese attention adversarial network. ; Steps 2-3: Based on the threshold H, signals greater than the threshold are considered abnormal, while signals below the threshold are considered normal. This is represented as follows: 。 2. The method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks as described in claim 1, characterized in that, In step 1-1, the log-Mel energy feature extraction of the original acoustic signal specifically includes pre-emphasis, framing, windowing, fast Fourier transform (FFT), and log-Mel filtering.

3. The method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks as described in claim 2, characterized in that, Step 1-1 is as follows: Step 1-1: For pre-weighting, by calculate, For input signal, This is the output after pre-emphasis. This is the pre-emphasis coefficient; Steps 1-2: For frames, the frame length is... Frame shift is ; Steps 1-3: For windowing, select a Hamming window ω(n), and calculate it using the formula x(n) = ω(n)s(n), where s(n) is the width of the window and is the nth sampling point of N; Steps 1-4: For the Fast Fourier Transform, convert the signals obtained from steps 1-1 to 1-3 to the frequency domain. The calculation formula is as follows: Where X is the input acoustic signal, and j is the imaginary unit. for The corresponding frequency domain signal, The number of points in the Fourier transform; Steps 1-5: For the Mel filter bank, the frequency domain information obtained in Step 1-4 is weighted to obtain the energy of each Mel frequency band. There are a total of M triangular filters. The frequency response H of the m-th Mel filter is then... m (k) is calculated using the formula, where f(m) is the center frequency: Steps 1-6: For the logarithmic Mel-frequency filtering operation, take the logarithm of the Mel-frequency energy obtained in Step 1-5 to obtain the logarithmic Mel-energy of each frame signal after filtering by the m-th Mel-filter. The calculation formula is as follows: 。 4. The method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks as described in any one of claims 1-3, characterized in that, Steps 1-3 are implemented as follows: Step 1-3.1: Pass the input X through a fully connected layer with n hidden units, and calculate the spectral attention weights of the input data using the sigmoid activation function. Normalize the weights to obtain the normalized spectral attention weights. W n,t Multiply by the input X to obtain the weighted features The calculation formula is as follows: Where X represents the spectral characteristics of the input acoustic signal, and V n and b n Let represent the weights and biases of the nth hidden unit in the fully connected layer, respectively. It is the number of Mel bins. Represents element-wise multiplication; Step 1-3.2: Extract the feature representations of the log-Mel energy of the source and target domains using GEF(x) respectively. s ) and GEF(x t ) is obtained, where GEF(.) represents the twin encoder.

5. The method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks as described in any one of claims 1-3, characterized in that, Steps 1-4 are implemented as follows: Step 1-4.1: Construct a domain discriminator and distinguish source and target features from the outputs of the two GEFs. The calculation formula is as follows: in, and These are the log-Mel energy characteristics of the source and target domains, respectively, W. D and W GEF These represent the learnable weight parameters of the discriminator D and the encoder GEF, respectively. Step 1-4.2: Align the feature representations of the source and target domains. The calculation formula is the maximum mean difference, which aims to match the data distributions of the source and target domains. Here, ||.|2 represents L2 regularization. in, and These are the log-Mel energy characteristics of the source and target domains, respectively, W. GEF This represents the learnable weight parameters of the encoder GEF. , These are the number of input log-Mel energy features in the source and target domains, respectively.

6. The method for detecting abnormal acoustic signals in industrial equipment based on Siamese attention adversarial networks as described in any one of claims 1-3, characterized in that, Steps 1-7 are as follows: Step 1-7.1: Process the training set according to Step 1-1 to obtain the log-Mel energy features of the source and target domains corresponding to the training set; Step 1-7.2: Input the features obtained in Step 1-7.1 into the Siamese Attention Adversarial Network (SAD). The SAD consists of GEF, GD, and D. GEF consists of a frequency domain-based attention mechanism module and 6 Dense layers, with batch normalization and ReLU activation function after each Dense layer. GD consists of 6 Dense layers, with the last Dense layer directly outputting the result. D consists of two Dense layers, with batch normalization, ELU activation, and Dropout regularization after each Dense layer. Steps 1-7.3: Optimize according to steps 1-6, and train the Siamese attention adversarial network. The anomaly score is the average of the sum of the absolute values ​​of the differences between normal samples and anomaly samples. The formula for calculating the anomaly score is as follows: Where, x k The log-Mel energy characteristic at time k. For the reconstructed signal features, K is the number of time frames. F and P represent the number of Mel filter banks and the number of context windows, respectively, and ||.|2 represents L2 regularization; Step 1-7.4: Determine the state of the industrial equipment based on the threshold H. The threshold is defined as follows: Let H = H * ... * H * H * H * H * H * H * H * H * H * H * H * H * H * H obey distributed, The parameters of the distribution are determined by Histogram estimation is given; the anomaly score detection threshold is determined as follows: The 90th percentile of the distribution is used to determine whether an abnormal score of a sound signal segment is greater than this threshold; otherwise, it is considered normal.

Images