Industrial equipment abnormal sound data expansion method based on improved generative adversarial network

Abstract

The invention relates to the field of acoustic signal processing, and discloses an industrial equipment abnormal sound data expansion method based on an improved generative adversarial network, and the method comprises the steps: constructing an improved generative adversarial network I-WaveGAN model; the I-WaveGAN model is trained, and a trained model is obtained; and generating the abnormal sound audio data of the industrial equipment by using the trained model. The method has the beneficial effects that high-fidelity modeling of audio features of industrial equipment is realized, the quality, diversity and discrimination of data expansion are remarkably improved, the modeling problem caused by abnormal sample deficiency is effectively relieved, and an efficient, stable and generalizable training data enhancement method is provided for an abnormal sound detection system.

Description

Technical Field

[0001] The present invention relates to the field of acoustic signal processing, and in particular to a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network. Background Art

[0002] Audio data augmentation technology transforms or synthesizes existing audio samples to generate more representative new audio data, thereby enhancing the model's training capabilities and improving its generalization performance in complex environments. This technology is of great significance in artificial intelligence applications such as speech recognition, abnormal sound detection, and speech synthesis. In particular, in the task of abnormal sound detection in industrial equipment, where abnormal data is scarce and expensive to obtain, audio data augmentation is a key means of improving model performance and alleviating data imbalance.

[0003] Currently, audio data augmentation methods can be primarily categorized into traditional signal processing methods and deep learning-based approaches. Traditional methods, including time shifting, time stretching, frequency masking, reverberation addition, and background noise superposition, can simulate some real-world environmental variations and, to a certain extent, increase the diversity of training samples. These methods are simple to operate and require relatively low resources, making them suitable for early system deployment and basic experimental verification. However, due to their reliance on fixed transformation rules, the authenticity and diversity of generated samples are limited, making them unable to fully reflect the complex sound characteristics of real industrial environments.

[0004] To improve the effectiveness of data augmentation, methods based on Generative Adversarial Networks (GANs) have emerged in recent years. These methods build an adversarial mechanism between a generator and a discriminator to learn the distribution of real audio data, thereby generating more diverse and realistic audio samples. Representative models, such as WaveGAN, generate audio waveforms directly in the time domain, effectively preserving the temporal structure of the signal and significantly improving audio generation quality compared to traditional methods.

[0005] In summary, audio data augmentation technology is evolving from traditional rule-based methods to adaptive generation methods centered on deep learning. This is particularly important for anomaly detection in industrial equipment audio. However, existing generation models still face challenges in handling industrial scenario characteristics such as high noise backgrounds, multiple device types, and long time series. Therefore, developing more stable and efficient audio generation mechanisms has become a hot research topic. Summary of the Invention

[0006] The purpose of the present invention is to propose a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network, so as to solve the technical problems that the existing audio data expansion methods still have many shortcomings in terms of authenticity, temporal expression, training stability and task adaptability.

[0007] Specifically, the present invention provides a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network, comprising the following steps:

[0008] S1. Build an improved generative adversarial network I-WaveGAN model;

[0009] S2. Training the I-WaveGAN model to obtain a trained model;

[0010] S3. Use the trained model to generate audio data of abnormal sounds of industrial equipment.

[0011] A storage medium stores instructions and data for implementing a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network.

[0012] A device for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network.

[0013] The beneficial effects provided by the present invention are:

[0014] 1. Enhanced generation stability: This paper introduces layer normalization and Dropout technology based on WaveGAN, which significantly improves the stability and convergence speed of the generator during training, and avoids the unstable performance of traditional batch normalization under long sequence and small batch conditions.

[0015] 2. Improved temporal feature modeling capabilities: By designing a temporal attention module in the discriminator, the model's ability to focus on key time steps is enhanced, enabling the generative model to more accurately capture highly recognizable temporal features in abnormal audio from industrial equipment.

[0016] 3. Dual optimization of training stability and discrimination ability: This paper introduces a spectral normalization mechanism in each convolutional layer of the discriminator to effectively suppress the gradient explosion and gradient vanishing problems, thereby improving the model's ability to discriminate between real and generated samples and the stability of the overall training.

[0017] 4. Stronger industrial adaptability: In view of the scarcity of abnormal audio and complex backgrounds in industrial scenarios, I-WaveGAN can generate higher-quality audio samples that are more consistent with the actual operating status of the equipment. This provides reliable training data support for industrial equipment anomaly detection tasks, and has stronger engineering adaptability and practical value.

[0018] Through the above technical solution, the present invention achieves high-fidelity modeling of the audio characteristics of industrial equipment, significantly improves the quality, diversity and discriminability of data expansion, effectively alleviates the modeling difficulties caused by the lack of abnormal samples, and provides an efficient, stable and scalable training data enhancement method for abnormal sound detection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a simple flow chart of the method of the present invention;

[0020] Figure 2 Schematic diagram of the I-WaveGAN model structure of the present invention;

[0021] Figure 3 It is a schematic diagram of the layer normalization mechanism;

[0022] Figure 4 It is a schematic diagram of the Dropout mechanism;

[0023] Figure 5 This is a schematic diagram of the temporal attention mechanism.

[0024] Figure 6 It is a schematic diagram of the working of the hardware device of an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0026] Before formally explaining the present invention, the scheme of the present invention is first generally explained for easy understanding.

[0027] Please refer to Figure 1 The present invention provides a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network, comprising:

[0028] S1. Build an improved generative adversarial network I-WaveGAN model;

[0029] It should be noted that the present invention improves the WaveGAN model, mainly in the following aspects:

[0030] (1) Layer normalization is introduced in the generator to replace the original batch normalization, and a Dropout layer is added to improve the training stability and generation ability of the generator;

[0031] (2) A temporal attention module is introduced into the discriminator to enhance the model’s attention to key features when processing long time series data;

[0032] (3) Spectral normalization technology is added to the discriminator to improve the stability of the discriminator and avoid the gradient explosion problem.

[0033] The above series of improvements converge into the I-WaveGAN (Improved-WaveGAN) model.

[0034] Please refer to Figure 2 , Figure 2 Schematic diagram of the structure of the I-WaveGAN model of the present invention.

[0035] The improved generative adversarial network I-WaveGAN model described in step S1 introduces layer normalization and Dropout mechanisms in the generator.

[0036] The generator includes: a fully connected layer, a layer normalization mechanism, an activation function, a Dropout mechanism and a transposed convolutional layer.

[0037] The random noise vector enters the fully connected layer, layer normalization mechanism, ReLU activation function, transposed convolution layer, Dropout mechanism, ReLu activation function, and Tanh activation function in sequence to obtain the generated audio.

[0038] As an example, the generator works as follows:

[0039] The generator takes as input a random noise vector z. It first maps the noise vector to a higher-dimensional space through a fully connected layer. Then, multiple convolutional transpose layers are used to amplify the data layer by layer, gradually generating the target audio. The output of each layer of the generator undergoes layer normalization and dropout to improve training stability and prevent overfitting. The final output of the generator is an audio signal, which is processed by the Tanh activation function before output to ensure that the generated audio signal has a value range between [-1, 1], meeting the standardization requirements for audio signals.

[0040] It should be noted that the discriminator includes: a one-dimensional convolution block, a temporal attention module and a flattening layer.

[0041] The one-dimensional convolution block includes: a spectral normalization mechanism, a one-dimensional convolution layer and a LeakyReLU activation function.

[0042] The audio generated by the generator passes through the first one-dimensional convolution block, the second one-dimensional convolution block, the first temporal attention module, the third one-dimensional convolution block, the fourth one-dimensional convolution block, the second temporal attention module, the fifth one-dimensional convolution block, the sixth one-dimensional convolution block and the flattening layer in sequence, and then is output through the full connection to obtain the judgment result.

[0043] As an example, the input to the discriminator can be audio data generated by the generator or real audio data. The input data passes through multiple convolutional layers, all of which use spectral normalization to stabilize the training process. The output of each layer passes through a Leaky ReLU activation function. Furthermore, to enhance the discriminator's focus on important time points in the audio signal, a temporal attention mechanism is introduced into the discriminator.

[0044] S2. Training the I-WaveGAN model to obtain a trained model;

[0045] It should be noted that in the I-WaveGAN model, the training process adopts an alternating optimization strategy of the generator and the discriminator, so that the generator and the discriminator constantly compete with each other during the training process, ultimately achieving the goal of generating high-quality audio data.

[0046] Specifically, during training, the generator and discriminator are alternately optimized, continuously improving their respective capabilities through adversarial training. The generator is able to gradually produce higher-quality, more realistic audio, while the discriminator continuously strengthens its ability to distinguish real from fake audio. Ultimately, after multiple iterations, the generator is able to generate audio signals that closely resemble real audio data, thus completing the task of generating high-quality audio data.

[0047] S3. Use the trained model to generate audio data of abnormal sounds of industrial equipment.

[0048] In the I-WaveGAN model, the generator is improved in two aspects: replacing batch normalization with layer normalization and introducing a dropout layer. These improvements are intended to improve the training stability of the generator. This paper describes these improvements in more detail as follows:

[0049] (1) Introduction of layer normalization: In the original WaveGAN model, the generator uses batch normalization for regularization. Batch normalization eliminates internal covariate shift by standardizing the activation values of each layer, thereby accelerating training. However, when processing long time series data such as audio, batch normalization may be affected by batch size and data order. Especially in long sequence training, the effect of batch normalization is not as expected. In order to solve this problem, the present invention replaces batch normalization with layer normalization. The biggest advantage of layer normalization is that it independently normalizes each sample at each time step, rather than normalizing across the entire batch, such as Figure 3 shown.

[0050] Specifically, layer normalization normalizes the activation values of each layer using formula (1).

[0051]

[0052] Where, Denotes the i-th input sample, μ and σ are the mean and standard deviation of the input samples, respectively, ∈ is a small constant to prevent division by zero errors, and γ and β are learnable parameters that control scaling and translation, respectively. Layer normalization normalizes the activation values at each time step, making the generator more stable when processing long time series data and avoiding the negative impact of batch size changes.

[0053] In addition, layer normalization helps eliminate the problem of gradient disappearance or explosion in deep networks. For audio data generation tasks, the introduction of layer normalization improves the training stability of the generator, making it more adaptable in learning complex audio patterns.

[0054] (2) Introducing Dropout layer: In order to improve the generalization ability of the generator and reduce overfitting, I-WaveGAN introduces Dropout layer in each layer of the generator. As shown in the figure below, Dropout is a commonly used regularization method. By randomly "discarding" the connections of some neurons during the training process, it reduces the model's dependence on certain specific features and prevents the model from overfitting the training data. Figure 4 shown.

[0055] The basic principle of Dropout is to randomly select some neurons according to a given probability (i.e., the retention probability) in each training iteration and temporarily set their output to zero. Assuming that the output of the neuron is h, the output after the Dropout operation is shown in Equation (2).

[0056]

[0057] Here, Bernoulli(p) represents a Bernoulli random variable generated with probability p, and p is the retention probability, which is the probability that a neuron is retained. This means that during each training session, only a subset of neurons participate in the computation, forcing the model to be independent of certain features and enhancing its generalization ability.

[0058] In the I-WaveGAN generator, dropout is applied to each transposed convolutional layer, and the dropout ratio is gradually reduced as training progresses. Specifically, starting from a dropout probability of 0.95 and gradually decreasing to 0.8, this gradual attenuation strategy ensures that the generator learns features more freely in the early stages of training, while reducing the dropout ratio in the later stages stabilizes the model's learning process and prevents overfitting.

[0059] In addition, the discriminator has been improved in two aspects: the introduction of a temporal attention module and spectral normalization technology. These improvements enable the discriminator to more accurately identify the difference between fake samples produced by the generator and real samples, thereby improving the stability and training efficiency of the entire adversarial network. Detailed descriptions of each improvement are as follows:

[0060] (1) Temporal attention module: In the original WaveGAN discriminator, the convolutional layer uses equalization processing on the features of all time steps of the audio signal, which makes the discriminative features of key temporal events easily drowned out by background noise. In order to enhance the model's perception of key temporal areas, this paper designs a lightweight temporal attention module. Through the dynamic feature calibration mechanism, the discriminator can autonomously focus on audio segments with high discrimination, thereby improving the game efficiency of adversarial training. Its model structure is as follows: Figure 5 shown.

[0061] The core idea of the temporal attention mechanism is to control what the network focuses on when processing time series data by assigning a weight to each time step, so that the network can focus on the time region that is most important to the result in generation or discrimination tasks.

[0062] In this invention, the input signal of the temporal attention module is First, the number of channels of the input signal is compressed through a one-dimensional convolution operation. Assuming the compression ratio is r, the intermediate result obtained by convolution has a lower number of channels (C / r), as shown in Equation (3).

[0063] Z1=Conv1D(X,W1)#(3)

[0064] Where W1 is the convolution kernel. The goal of the convolution operation is to compress the number of channels of the input signal through a small convolution kernel. The output of the convolution operation is Next, the compressed signal is processed through another layer of convolution to capture the signal's temporal dependencies. The goal is to learn the attention weight for each time step, that is, the influence of each time point on the overall information. To enable the convolution to output a weight for each time step, a convolution operation with an activation function is used, as shown in Equation (4).

[0065] Z2=σ(Conv1D(Z1,W2))#(4)

[0066] Where W2 is the convolution kernel used to learn temporal dependencies, and the output Z2 is a tensor of shape B×T×1, which represents the attention weight of each time step. Finally, the learned temporal attention weight Z2 is used to weight the original input signal, thereby highlighting the information of important time steps. By multiplying the weight matrix Z2 with the original input X, the weighted output signal is obtained. As shown in formula (5).

[0067]

[0068] Where ⊙ represents element-by-element multiplication, and Z2 is aligned with the input after expansion along the channel dimension. Through the above steps, the temporal attention module can assign different weights to each time step, allowing the model to focus on the most important temporal features.

[0069] (2) Spectral Normalization: Another improvement in the discriminator is the introduction of spectral normalization technology, which aims to prevent the gradient explosion or gradient vanishing problem in the network and improve the stability of the discriminator during training. Spectral normalization technology makes the weight update of the network more stable by normalizing the weight matrix of each layer to the maximum singular value, avoiding unstable training caused by excessive gradients. The key idea of spectral normalization is to limit the expressive power of the network by normalizing the weight matrix W of each layer to the maximum singular value. Specifically, assuming is the weight matrix of a layer in the discriminator, the weight matrix after spectral normalization It can be expressed as formula (7).

[0070]

[0071] Where σ(W) is the maximum singular value of the matrix W, as shown in Equation (8).

[0072]

[0073] Where, σ i (W) represents the i-th singular value of the matrix W. After normalization The maximum singular value of is limited to 1, ensuring that the weight updates of each layer of the network remain within an appropriate range. This regularization method not only helps control the expressiveness of the network and prevent gradient explosion, but also ensures the stability of the network weight updates during training by limiting the influence of the weights of each layer.

[0074] See Figure 6 , Figure 64 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically includes: an industrial equipment abnormal sound data expansion device 401 based on an improved generative adversarial network, a processor 402 and a storage medium 403.

[0075] An abnormal sound data expansion device 401 for industrial equipment based on an improved generative adversarial network: The abnormal sound data expansion device 401 for industrial equipment based on an improved generative adversarial network implements the abnormal sound data expansion method for industrial equipment based on an improved generative adversarial network.

[0076] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network.

[0077] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network.

[0078] The beneficial effects of the present invention are:

[0079] 1. Enhanced generation stability: This paper introduces layer normalization and Dropout technology based on WaveGAN, which significantly improves the stability and convergence speed of the generator during training, and avoids the unstable performance of traditional batch normalization under long sequence and small batch conditions.

[0080] 2. Improved temporal feature modeling capabilities: By designing a temporal attention module in the discriminator, the model's ability to focus on key time steps is enhanced, enabling the generative model to more accurately capture highly recognizable temporal features in abnormal audio from industrial equipment.

[0081] 3. Dual optimization of training stability and discrimination ability: This paper introduces a spectral normalization mechanism in each convolutional layer of the discriminator to effectively suppress the gradient explosion and vanishing problems, thereby improving the model's ability to discriminate between real and generated samples and the stability of the overall training.

[0082] 4. Stronger industrial adaptability: In view of the scarcity of abnormal audio and complex backgrounds in industrial scenarios, I-WaveGAN can generate higher-quality audio samples that are more consistent with the actual operating status of the equipment. This provides reliable training data support for industrial equipment anomaly detection tasks, and has stronger engineering adaptability and practical value.

[0083] Through the above technical solution, the present invention achieves high-fidelity modeling of the audio characteristics of industrial equipment, significantly improves the quality, diversity and discriminability of data expansion, effectively alleviates the modeling difficulties caused by the lack of abnormal samples, and provides an efficient, stable and scalable training data enhancement method for abnormal sound detection systems.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network, characterized by: The following steps are involved: S1. Build an improved generative adversarial network I-WaveGAN model; S2. Training the I-WaveGAN model to obtain a trained model; S3. Use the trained model to generate audio data of abnormal sounds of industrial equipment.

2. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 1, characterized in that: The improved generative adversarial network I-WaveGAN model described in step S1 introduces layer normalization and Dropout mechanisms in the generator.

3. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 1, characterized in that: The improved generative adversarial network I-WaveGAN model described in step S1 adds a temporal attention module and a spectral normalization mechanism to the discriminator.

4. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 2, characterized in that: The generator includes: a fully connected layer, a layer normalization mechanism, an activation function, a Dropout mechanism and a transposed convolutional layer.

5. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 4, characterized in that: The random noise vector enters the fully connected layer, layer normalization mechanism, ReLU activation function, transposed convolution layer, Dropout mechanism, ReLu activation function, and Tanh activation function in sequence to obtain the generated audio.

6. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 3, characterized in that: The discriminator includes: a one-dimensional convolution block, a temporal attention module and a flattening layer.

7. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 6, characterized in that: The one-dimensional convolution block includes: a spectral normalization mechanism, a one-dimensional convolution layer and a LeakyReLU activation function.

8. The method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network according to claim 6, characterized in that: The audio generated by the generator passes through the first one-dimensional convolution block, the second one-dimensional convolution block, the first temporal attention module, the third one-dimensional convolution block, the fourth one-dimensional convolution block, the second temporal attention module, the fifth one-dimensional convolution block, the sixth one-dimensional convolution block and the flattening layer in sequence, and then is output through the full connection to obtain the judgment result.

9. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network as described in any one of claims 1 to 8.

10. A device for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network, characterized by: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a method for expanding abnormal sound data of industrial equipment based on an improved generative adversarial network as described in any one of claims 1 to 8.

Images