Method for generating audio deep learning training data based on adversarial neural network

Abstract

The invention relates to a method for generating audio deep learning training data based on an adversarial neural network (GAN), which is a data synthesis technology based on a generative adversarial network (GAN) and is used for generating audio training samples of scarce categories so as to effectively expand the diversity of a data set. According to the method, the generation capability of the GAN model is utilized to simulate the audio samples of the rare category, and the problem that the samples of the rare category are insufficient in a traditional data set can be solved. Through the technology, researchers can reduce the cost of manually collecting samples, the consumption of human resources is greatly reduced, and the universality of the data set in category coverage is ensured at the same time. In addition, the generated diversified samples can enhance the generalization ability of the model, so that the model is more stable when processing changeful data in the real world. According to the method, the training efficiency of the model can be effectively improved, the robustness of the model in different scenes can be improved, and the precision and practicability of an audio classification task in specific field application are further promoted.

Description

Technical Field

[0001] This invention relates to a method for generating audio deep learning training data based on adversarial neural networks, belonging to the field of computer multimedia (image and video) information processing technology. Background Technology

[0002] In deep learning tasks for audio classification, a large number of audio training samples of different categories are required. Currently, the most commonly used methods for collecting training datasets include the following: 1. Download publicly available datasets from the internet: Commonly used datasets include UrbanSound8K, AudioSet, ESC-50, LibriSpeech, GTZAN, etc.

[0003] 2. Web scraping: Using web crawler technology to scrape publicly available audio content from internet platforms, such as video websites, music platforms, and podcasts. This usually requires compliance with copyright laws and obtaining the necessary licenses.

[0004] 3. Manual collection and labeling.

[0005] The existing methods for collecting audio classification training datasets have the following main problems: 1. Limitations of Public Datasets: While existing public audio datasets on the internet offer large sample sizes and broad diversity, they are often too generalized to directly meet the needs of specific tasks. Therefore, using public datasets often requires secondary screening and cleaning to ensure the relevance and quality of the audio samples, a process that consumes significant human and time resources. Furthermore, for niche or specific domain audio, such as medical diagnostics, industrial noise, or dialects of specific languages, public datasets often lack sufficient coverage and cannot effectively support model training for these tasks.

[0006] 2. Compliance and Security Issues of Web Scraping: Scraping audio data from the web presents certain security risks and copyright compliance issues. Different platforms have varying authorization methods and usage restrictions for audio data, making it difficult to obtain legal usage rights and potentially involving complex legal compliance requirements. Furthermore, scraped audio data is often unwashed and may contain irrelevant content, low-quality segments, or privacy-sensitive information, requiring further processing and annotation before it can be used for model training. This process increases the complexity of data processing and resource investment.

[0007] 3. Data Quantity and Category Scarcity: Deep learning models typically require a large amount of training data, especially in audio classification tasks where the demand for samples from specific categories is even greater. However, audio data for specific domains or rare categories is often very scarce, requiring manual collection. This not only increases time costs but is also inefficient, making it difficult to quickly meet training needs. Therefore, insufficient data has become a significant bottleneck for improving model performance, limiting the generalization ability and application effectiveness of deep learning models. Summary of the Invention

[0008] This invention proposes a method for generating audio deep learning training data based on adversarial neural networks. Its purpose is to overcome the above-mentioned shortcomings of existing technologies and generate a large number of samples of scarce categories while ensuring low cost and high training efficiency.

[0009] The technical solution of this invention: a method for generating audio deep learning training data based on adversarial neural networks, comprising four modules: a data management module, a data processing module, a model training module, and a database module, wherein... The data management module has two main functions: first, it manages category sample data, selecting target category samples as initial data; second, it receives expanded data and inputs it into vector and relational databases for rapid filtering and searching in subsequent tasks. The data processing module has three main functions: First, it performs format conversion on the initial data, including sampling rate adjustment and frame segmentation, to ensure data consistency. Second, it extracts features from the data, converting the audio into a format suitable for model processing, including Mel spectrograms and Mel frequency cepstral coefficients, to reduce the computational burden on the model. These feature maps are used as input to the GAN model to generate higher-quality audio samples. Third, it performs post-processing on the data, restoring the feature maps to audio, evaluating the quality of the generated audio samples, removing low-quality samples, and integrating high-quality synthetic audio samples with the original dataset to enrich the diversity of the training dataset. The model training module is used to alternately train the generator and discriminator: In each round of training, the discriminator weights are fixed first, and the generator is trained to generate audio samples and try to "deceive" the discriminator. "Deceiving" means increasing the probability that the discriminator judges the generated samples as real samples. Then, the generator weights are fixed, and the discriminator is trained to distinguish between generated samples and real samples. This allows the generator to continuously improve the realism of the generated samples, and the discriminator to continuously improve its ability to distinguish between generated samples and real samples. The database module is divided into two parts: a relational database and a vector database. The relational database is used to maintain data including sample categories and model evaluation scores; the vector database is used to store audio feature vectors for subsequent similarity searches.

[0010] Preferably, it includes the following steps: Step 1, data preprocessing, including: Sample preparation: Select target category samples from existing audio datasets as initial data, and perform format conversion, including sampling rate adjustment and frame segmentation, to ensure data consistency. Feature extraction: Audio is converted into Mel frequency cepstral coefficients to reduce the computational burden on the model. These feature maps are used as input in the GAN model to generate higher quality audio samples. Step 2: Construct and generate the GAN model, including: Generator Design: The generator is used to generate new audio samples. The generator model receives random noise or a conditional vector as input, processes it through a multi-layer convolutional neural network or other neural network architecture, and generates realistic audio feature maps. Discriminator Design: The discriminator is used to distinguish between generated samples and real samples. The discriminator evaluates the input audio feature map and determines whether the sample is generated by the generator or comes from the real dataset. The GAN model gradually generates more realistic audio samples by continuously improving the capabilities of the generator and discriminator. Step 3, Model Training, including: Alternate training of the generator and discriminator: In each round of training, first, the discriminator weights are fixed, and the generator is trained to generate audio samples and attempt to "deceive" the discriminator. "Deceiving" means increasing the probability that the discriminator will classify the generated samples as real samples. Then, the generator weights are fixed again, and the discriminator is trained to distinguish between generated and real samples. This process allows the generator to continuously improve the realism of its generated samples, and the discriminator to continuously improve its ability to distinguish between generated and real samples. Loss function design: Wasserstein distance loss is used to further optimize the performance of the generator and discriminator to ensure the quality of the generated audio; Step 4, post-processing, including: Feature Map Reconstruction into Audio: The generated feature map is reconstructed into waveform audio using inverse Mel-frequency transform. Quality assessment and screening: The generated audio samples are quality assessed and low-quality samples are removed to ensure the quality of the final dataset. The assessment methods include computer scoring and human auditory assessment. Computer scoring includes signal feature similarity index. Step 5: Integrate the samples into the training dataset, including: Data augmentation and integration: High-quality synthetic audio samples are integrated with the original dataset to enrich the diversity of the training dataset. Simultaneously, data augmentation is performed on the generated samples, including adding slight noise and adjusting pitch, to further increase diversity. Model Training and Validation: The model was trained using the augmented dataset to examine the impact of generated samples on model performance. By comparing the performance of models with and without synthetic samples, the effectiveness of GAN-generated samples in improving the model's generalization ability and robustness was verified. Step 6: Set the model inference criteria, and iterate through sample supplementation and training. Exit when the model score reaches the preset criteria.

[0011] Preferably, the alternating training of the generator and discriminator in step 3, i.e., the joint training method of the generator and discriminator, is specifically designed to include: Network architecture design: The generator is designed as a custom convolutional neural network, taking a random noise vector \(z\) as input, i.e., Gaussian noise, and an optional conditional vector, i.e., audio category information, to generate an audio feature map in the form of a Mel spectrogram. The generator's structure is represented as follows: Input noise vector (batch_size, 100) Fully connected layer -> Shape converted to (batch_size, 512, 4, 1) Transposed convolutional layer (512, 4, 1) -> (256, 8, 1) + ReLU + Batch Normalization Transposed convolutional layer (256, 8, 1) -> (128, 16, 1) + ReLU + Batch Normalization Transposed convolutional layer (128, 16, 1) -> (64, 32, 1) + ReLU + Batch Normalization Transposed convolutional layer (64, 32, 1) -> (32, 64, 1) + ReLU + Batch Normalization Transposed convolutional layer (32, 64, 1) -> (output_channels, 128, 1) + Tanh; The discriminator employs a custom deep convolutional neural network, taking as input real audio feature maps and synthetic audio feature maps generated by the generator, and outputting a real value representing the probability of a sample being a real sample. To describe the transformation from input data to output probability, the discriminator's structure is represented as follows: Input audio data (batch_size, 128, 1) One-dimensional convolutional layer (32 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 64, 32) One-dimensional convolutional layer (64 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 32, 64) One-dimensional convolutional layer (128 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 16, 128) One-dimensional convolutional layer (256 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 8, 256) Flatten Fully connected layer -> Single output + Sigmoid activation -> (batch_size, 1); Joint training strategy: Gradually update the weights of the generator and discriminator, and optimize the generator and discriminator separately in each training iteration to improve the stability of training and the realism of generated samples; Iterative training steps: For discriminator training, the generator weights are fixed, and the discriminator is trained using real and generated audio feature maps. The discriminator optimizes its weights by calculating the distinguishing power between real and generated samples, as shown in Equation 1. The Wasserstein distance loss function is used to improve the discriminator's ability to distinguish real samples. The generator is trained by fixing the discriminator weights and generating new audio feature maps. The goal is to make these feature maps "deceive" the discriminator as much as possible; "deceiving" means increasing the probability that the discriminator will classify the generated samples as real samples. The generator uses Wasserstein loss as its loss function and incorporates feedback from the discriminator to continuously adjust the weights to generate higher-quality audio feature maps. For real sample xx and generated sample G(z), the loss function is: Formula 1: L D =E x∼pdata [D(x)]−E z∼pz [D(G(z))] Where D(x) represents the score of the discriminator for the real sample, and D(G(z)) represents the score for the generated sample.

[0012] Preferably, the loss function design in step 3 includes loss function design and improvement: the loss functions of the generator and discriminator are designed as adversarial losses, wherein the generator's loss is to maximize the probability that the generated sample is judged as a real sample by the discriminator, while the discriminator's loss is to improve the distinction between real and generated samples; to stabilize the training process, a feature matching loss is introduced into the generator; this loss requires the generator to not only generate samples that the discriminator cannot distinguish, but also generate samples that are more similar to real sample features; gradient penalty is added to Wasserstein GAN to prevent the discriminator from overfitting and improve the stability of the model.

[0013] Preferably, step 3 further includes model evaluation and tuning: after each round of training, the quality of the generated samples is evaluated using signal feature evaluation, including signal-to-noise ratio, feature similarity, and human hearing tests, and model weights with better generation performance are selected; the learning rates of the generator and discriminator are adaptively adjusted according to the model's loss convergence to accelerate model training and improve the quality of the generated audio.

[0014] Preferably, in step 3, the optimization steps for the generator and discriminator in the joint training strategy include: Sampling real samples: from the real data distribution p data A batch of real samples x were sampled from the middle; Sample generation: from noise distribution p z The noise z is sampled and then generated into a sample G(z) by a generator G; Calculate the loss: Use the Wasserstein loss function L described above. D Calculate the loss of the discriminator; Backpropagation: The gradient of the loss with respect to the discriminator parameters is calculated using the backpropagation algorithm; Update weights: Update the discriminator weights using the Adam optimization algorithm; Weight pruning: After updating the weights, ensure that the discriminator's weights are within the preset range; The generator maximizes the output of the discriminator: L G =−E z∼pz [D(G(z))] That is, the generator hopes that the generated samples will get higher scores, so as to approximate the distribution of the real samples; Termination condition for joint training of generator and discriminator: The training process terminates when one of the following conditions is met: 1. The generated audio samples meet the preset quality standards in signal characteristics or auditory tests; 2. The loss functions of the generator and discriminator are basically stable over multiple cycles, indicating that the model has reached convergence.

[0015] The advantages of this invention are: The method is rationally designed and can be used to generate a large number of audio training samples for scarce categories, effectively expanding the diversity of the dataset and ensuring broad category coverage. It allows for model training for specific scenarios, reducing the model's reliance on a few categories, mitigating data imbalance, and improving the model's generalization ability. It reduces the cost of researchers relying on manual sample collection, effectively lowering human resource consumption. The diverse samples generated enhance the model's generalization ability, making it more robust when dealing with highly variable real-world data. The use of automatically supplemented samples in model training reduces the complex data processing steps of traditional model training, improving training efficiency. It also improves the model's robustness in different scenarios, further advancing the accuracy and practicality of audio classification tasks in specific domain applications. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method for generating audio deep learning training data based on adversarial neural networks according to the present invention.

[0017] Figure 2 This is a flowchart of the method for generating audio deep learning training data based on adversarial neural networks according to the present invention.

[0018] Figure 3 This is a schematic diagram of the network structure of an embodiment of the generator in the method for generating audio deep learning training data based on adversarial neural networks according to the present invention.

[0019] Figure 4 This is a schematic diagram of the network structure of an embodiment of the discriminator in the method for generating audio deep learning training data based on adversarial neural networks of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to embodiments and specific implementation methods.

[0021] A method for generating deep learning audio training data based on Generative Adversarial Networks (GANs) is presented. This method utilizes GAN models to generate audio training samples representing rare categories, effectively expanding the diversity of datasets. By leveraging the generative capabilities of GAN models, it simulates audio samples representing rare categories, addressing the problem of insufficient rare category samples in traditional datasets. This technique allows researchers to reduce the cost of relying on manual sample collection, significantly decreasing human resource consumption, while ensuring broad category coverage of the dataset.

[0022] Furthermore, the generated diverse samples enhance the model's generalization ability, making it more robust when dealing with diverse real-world data. This method not only effectively improves the model's training efficiency but also enhances its robustness across different scenarios, further advancing the accuracy and practicality of audio classification tasks in specific domain applications.

[0023] Specifically, such as Figure 1 As shown, it comprises four modules: data management module, data processing module, model training module, and database module. The data management module has two main functions: first, it manages category sample data to facilitate the selection of target category samples as initial data; second, it receives expanded data and inputs it into vector databases and relational databases to facilitate rapid filtering and searching in subsequent tasks.

[0024] The data processing module has three main components: First, it performs format conversion on the initial data (such as sampling rate adjustment and frame segmentation) to ensure data consistency. Second, it extracts features from the data. To reduce the computational burden on the model, the audio is converted into a form suitable for model processing, such as Mel spectrograms, MFCC (Mel frequency cepstral coefficients), or other acoustic feature maps. These feature maps are used as inputs in the GAN model, which helps generate higher-quality audio samples. Third, it performs post-processing on the data, restoring the feature maps to audio, evaluating the quality of the generated audio samples, removing low-quality samples, and integrating high-quality synthetic audio samples with the original dataset to enrich the diversity of the training dataset.

[0025] The model training module primarily involves alternating training of the generator and discriminator: In each round of training, the discriminator weights are first fixed, and the generator is trained to generate audio samples and attempt to "fool" the discriminator; then, the generator weights are fixed again, and the discriminator is trained to distinguish between generated and real samples. Through this adversarial training, the generator continuously improves the realism of its generated samples, while the discriminator continuously enhances its ability to distinguish between generated and real samples.

[0026] The database module is mainly divided into two parts: a relational database and a vector database. The relational database mainly maintains data such as sample categories and model evaluation scores; the vector database stores audio feature vectors, facilitating subsequent similarity searches.

[0027] like Figure 2 As shown, the specific data flow is as follows: Step 1: Data Preprocessing Sample preparation: Select target category samples from existing audio datasets as initial data and perform format conversion (such as sampling rate adjustment, frame segmentation, etc.) to ensure data consistency.

[0028] Feature extraction: To reduce the computational burden on the model, the audio is converted into MFCC (Mel-frequency cepstral coefficients). These feature maps are used as input to the GAN model, which helps generate higher-quality audio samples.

[0029] Step 2: Construct a Generative Adversarial Network (GAN) model Generator Design: A generator is used to generate new audio samples. The generator model takes random noise or a conditional vector as input, processes it through a multi-layer convolutional neural network (CNN) or other neural network architecture, and generates realistic audio feature maps.

[0030] Discriminator Design: The discriminator distinguishes between generated samples and real samples. It evaluates the input audio feature maps, determining whether they are samples generated by the generator or from a real dataset. By continuously improving the capabilities of the generator and discriminator, the GAN model gradually generates more realistic audio samples.

[0031] Step 3: Model Training Alternating training of the generator and discriminator: In each round of training, the discriminator weights are first fixed, and the generator is trained to generate audio samples and attempt to "fool" the discriminator. Then, the generator weights are fixed again, and the discriminator is trained to distinguish between generated and real samples. Through this adversarial training, the generator continuously improves the realism of its generated samples, while the discriminator continuously improves its ability to distinguish between generated and real samples.

[0032] Loss function design: To ensure the quality of the generated audio, Wasserstein distance loss is used to further optimize the performance of the generator and discriminator.

[0033] Step 4, Post-processing Feature map reconstruction into audio: The generated feature map is reconstructed into waveform audio using inverse Mel spectrum transformation.

[0034] Quality assessment and screening: The generated audio samples are assessed for quality, and low-quality samples are removed to ensure the quality of the final dataset. Common assessment methods include computer scoring (such as signal feature similarity metrics) and human auditory assessment.

[0035] Step 5: Integrate the samples into the training dataset Data augmentation and integration: High-quality synthetic audio samples are integrated with the original dataset to enrich the diversity of the training dataset. Simultaneously, data augmentation can be performed on the generated samples by adding slight noise, adjusting pitch, etc., to further increase diversity.

[0036] Model Training and Validation: The model was trained using the augmented dataset to examine the impact of generated samples on model performance. By comparing the performance of models with and without synthetic samples, the effectiveness of GAN-generated samples in improving the model's generalization ability and robustness was verified.

[0037] Step 6: Set the model inference criteria, and iterate through sample supplementation and training. Exit when the model score reaches the preset criteria.

[0038] The specific design of the alternating training method for the generator and discriminator in step 3, i.e., the joint training method for the generator and discriminator, includes: Network architecture design: The generator (G) is designed as a custom convolutional neural network that takes a random noise vector \(z\) (Gaussian noise) and an optional conditional vector (audio category information) as input to generate an audio feature map in the form of a Mel spectrogram.

[0039] like Figure 3 As shown, the structure of the generator can be represented as follows: Input noise vector (batch_size, 100) Fully connected layer -> Shape converted to (batch_size, 512, 4, 1) Transposed convolutional layer (512, 4, 1) -> (256, 8, 1) + ReLU + Batch Normalization Transposed convolutional layer (256, 8, 1) -> (128, 16, 1) + ReLU + Batch Normalization Transposed convolutional layer (128, 16, 1) -> (64, 32, 1) + ReLU + Batch Normalization Transposed convolutional layer (64, 32, 1) -> (32, 64, 1) + ReLU + Batch Normalization Transposed convolutional layer (32, 64, 1) -> (output_channels, 128, 1) + Tanh The discriminator (D) uses a custom deep convolutional neural network, takes a real audio feature map as input and a synthetic audio feature map generated by the generator, and outputs a real value representing the probability of whether a sample is a real sample.

[0040] Figure 4 This is a schematic diagram of the discriminator's network structure, used to describe the transformation from input data to output probabilities: Input audio data (batch_size, 128, 1) One-dimensional convolutional layer (32 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 64, 32) One-dimensional convolutional layer (64 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 32, 64) One-dimensional convolutional layer (128 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 16, 128) One-dimensional convolutional layer (256 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 8, 256) Flatten Fully connected layer -> Single output + Sigmoid activation -> (batch_size, 1) Joint training strategy: Gradually update the weights of the generator and discriminator, and optimize the generator and discriminator separately in each training iteration to improve the stability of training and the realism of generated samples.

[0041] Iterative training steps: For discriminator training, the generator weights are fixed, and the discriminator is trained using both real and generated audio feature maps. The discriminator optimizes its weights by calculating the discriminative power between real and generated samples (see Equation 1). The Wasserstein distance loss function is used to improve the discriminator's ability to distinguish between real samples.

[0042] The generator is trained by fixing the discriminator weights and generating new audio feature maps that are designed to "deceive" the discriminator as much as possible (increasing the probability that the discriminator classifies the generated samples as real samples). The generator uses Wasserstein loss as its loss function and incorporates feedback from the discriminator to continuously adjust the weights to generate higher-quality audio feature maps.

[0043] For real sample xx and generated sample G(z), the loss function can be written as: Formula 1: L D =E x∼pdata [D(x)]−E z∼pz [D(G(z))] Where D(x) represents the score of the discriminator for the real sample, and D(G(z)) represents the score for the generated sample.

[0044] Step 3 involves loss function design, including its design and improvement: the loss functions for both the generator and discriminator are designed as adversarial losses. The generator's loss maximizes the probability that generated samples are judged as real samples by the discriminator, while the discriminator's loss improves the distinguishability between real and generated samples. To stabilize the training process, a feature matching loss is introduced into the generator. This loss requires the generator to not only generate samples that the discriminator cannot distinguish but also samples whose features are more similar to real samples. Gradient penalty is added to the Wasserstein GAN to prevent the discriminator from overfitting and improve the model's stability.

[0045] Step 3 also includes model evaluation and tuning: After each round of training, the quality of the generated samples is evaluated using signal feature assessments (such as signal-to-noise ratio, feature similarity, etc.) and human hearing tests to select model weights with better generation performance. Based on the model's loss convergence, the learning rates of the generator and discriminator are adaptively adjusted to accelerate model training and improve the quality of the generated audio.

[0046] Step 3 involves jointly training the generator and discriminator in a coordinated training strategy. The optimization steps include: Sampling real samples: from the real data distribution p data A batch of real samples x were sampled from the middle.

[0047] Sample generation: from noise distribution p z The noise z is sampled and then generated as a sample G(z) by a generator G.

[0048] Calculate the loss: Use the Wasserstein loss function L described above. D Calculate the loss of the discriminator.

[0049] Backpropagation: The gradient of the loss with respect to the discriminator parameters is calculated using the backpropagation algorithm.

[0050] Update weights: Update the discriminator weights using the Adam optimization algorithm.

[0051] Weight pruning: After updating the weights, ensure that the discriminator's weights are within the preset range.

[0052] The generator maximizes the output of the discriminator: L G =−E z∼pz[D(G(z))] This means that the generator hopes to generate samples with higher scores, thereby approximating the distribution of real samples.

[0053] Termination condition for joint training of generator and discriminator: The training process terminates when one of the following conditions is met: 1. The generated audio samples meet the preset quality standards in signal characteristics or auditory tests.

[0054] 2. The loss functions of the generator and discriminator are basically stable over multiple cycles, indicating that the model has reached convergence.

[0055] The above methods employ a joint training approach for the generator and discriminator, along with a specific alternating training strategy, to optimize the performance of both the generator and discriminator. This includes improved loss function design and feature extraction techniques, which enhance the quality and realism of the generated audio samples. A complete training data generation system workflow is designed, supporting the expansion and integration of subsequent search components. An adversarial network design is also included.

[0056] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for generating audio deep learning training data based on adversarial neural networks, characterized in that, It comprises four modules: data management, data processing, model training, and database. The data management module has two main functions: first, it manages category sample data, selecting target category samples as initial data; second, it receives expanded data and inputs it into vector and relational databases for rapid filtering and searching in subsequent tasks. The data processing module has three main functions: First, it performs format conversion on the initial data, including sampling rate adjustment and frame segmentation, to ensure data consistency. Second, it extracts features from the data, converting the audio into a format suitable for model processing, including Mel spectrograms and Mel frequency cepstral coefficients, to reduce the computational burden on the model. These feature maps are used as input to the GAN model to generate higher-quality audio samples. Third, it performs post-processing on the data, restoring the feature maps to audio, evaluating the quality of the generated audio samples, removing low-quality samples, and integrating high-quality synthetic audio samples with the original dataset to enrich the diversity of the training dataset. The model training module is used to alternately train the generator and discriminator: In each round of training, the discriminator weights are fixed first, and the generator is trained to generate audio samples and try to "deceive" the discriminator. "Deceiving" means increasing the probability that the discriminator judges the generated samples as real samples. Then, the generator weights are fixed, and the discriminator is trained to distinguish between generated samples and real samples. This allows the generator to continuously improve the realism of the generated samples, and the discriminator to continuously improve its ability to distinguish between generated samples and real samples. The database module is divided into two parts: a relational database and a vector database. The relational database is used to maintain data including sample categories and model evaluation scores; the vector database is used to store audio feature vectors for subsequent similarity searches.

2. The method for generating audio deep learning training data based on an adversarial neural network as described in claim 1, characterized in that, Includes the following steps: Step 1, data preprocessing, including: Sample preparation: Select target category samples from existing audio datasets as initial data, and perform format conversion, including sampling rate adjustment and frame segmentation, to ensure data consistency. Feature extraction: Audio is converted into Mel frequency cepstral coefficients to reduce the computational burden on the model. These feature maps are used as input in the GAN model to generate higher quality audio samples. Step 2: Construct and generate the GAN model, including: Generator Design: The generator is used to generate new audio samples. The generator model receives random noise or a conditional vector as input, processes it through a multi-layer convolutional neural network or other neural network architecture, and generates realistic audio feature maps. Discriminator Design: The discriminator is used to distinguish between generated samples and real samples. The discriminator evaluates the input audio feature map and determines whether the sample is generated by the generator or comes from the real dataset. The GAN model gradually generates more realistic audio samples by continuously improving the capabilities of the generator and discriminator. Step 3, Model Training, including: Alternate training of the generator and discriminator: In each round of training, first, the discriminator weights are fixed, and the generator is trained to generate audio samples and attempt to "deceive" the discriminator. "Deceiving" means increasing the probability that the discriminator will classify the generated samples as real samples. Then, the generator weights are fixed again, and the discriminator is trained to distinguish between generated and real samples. This process allows the generator to continuously improve the realism of its generated samples, and the discriminator to continuously improve its ability to distinguish between generated and real samples. Loss function design: Wasserstein distance loss is used to further optimize the performance of the generator and discriminator to ensure the quality of the generated audio; Step 4, post-processing, including: Feature Map Reconstruction into Audio: The generated feature map is reconstructed into waveform audio using inverse Mel-frequency transform. Quality assessment and screening: The generated audio samples are quality assessed and low-quality samples are removed to ensure the quality of the final dataset. The assessment methods include computer scoring and human auditory assessment. Computer scoring includes signal feature similarity index. Step 5: Integrate the samples into the training dataset, including: Data augmentation and integration: High-quality synthetic audio samples are integrated with the original dataset to enrich the diversity of the training dataset. Simultaneously, data augmentation is performed on the generated samples, including adding slight noise and adjusting pitch, to further increase diversity. Model Training and Validation: The model was trained using the augmented dataset to examine the impact of generated samples on model performance. By comparing the performance of models with and without synthetic samples, the effectiveness of GAN-generated samples in improving the model's generalization ability and robustness was verified. Step 6: Set the model inference criteria, and iterate through sample supplementation and training. Exit when the model score reaches the preset criteria.

3. The method for generating audio deep learning training data based on an adversarial neural network as described in claim 2, characterized in that, The specific design of the alternating training method for the generator and discriminator in step 3, i.e., the joint training method for the generator and discriminator, includes: Network architecture design: The generator is designed as a custom convolutional neural network, taking a random noise vector \(z\) as input, i.e., Gaussian noise, and an optional conditional vector, i.e., audio category information, to generate an audio feature map in the form of a Mel spectrogram. The generator's structure is represented as follows: Input noise vector (batch_size, 100) Fully connected layer -> Shape converted to (batch_size, 512, 4, 1) Transposed convolutional layer (512, 4, 1) -> (256, 8, 1) + ReLU + Batch Normalization Transposed convolutional layer (256, 8, 1) -> (128, 16, 1) + ReLU + Batch Normalization Transposed convolutional layer (128, 16, 1) -> (64, 32, 1) + ReLU + Batch Normalization Transposed convolutional layer (64, 32, 1) -> (32, 64, 1) + ReLU + Batch Normalization Transposed convolutional layer (32, 64, 1) -> (output_channels, 128, 1) + Tanh; The discriminator employs a custom deep convolutional neural network, taking as input real audio feature maps and synthetic audio feature maps generated by the generator, and outputting a real value representing the probability of a sample being a real sample. To describe the transformation from input data to output probability, the discriminator's structure is represented as follows: Input audio data (batch_size, 128, 1) One-dimensional convolutional layer (32 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 64, 32) One-dimensional convolutional layer (64 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 32, 64) One-dimensional convolutional layer (128 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 16, 128) One-dimensional convolutional layer (256 filters, kernel=4, stride=2) + Leaky ReLU + BatchNormalization -> (batch_size, 8, 256) Flatten Fully connected layer -> Single output + Sigmoid activation -> (batch_size, 1); Joint training strategy: Gradually update the weights of the generator and discriminator, and optimize the generator and discriminator separately in each training iteration to improve the stability of training and the realism of generated samples; Iterative training steps: For discriminator training, the generator weights are fixed, and the discriminator is trained using real and generated audio feature maps. The discriminator optimizes its weights by calculating the distinguishing power between real and generated samples, as shown in Equation 1. The Wasserstein distance loss function is used to improve the discriminator's ability to distinguish real samples. The generator is trained by fixing the discriminator weights and generating new audio feature maps. The goal is to make these feature maps "deceive" the discriminator as much as possible. "Deceiving" means increasing the probability that the discriminator will classify the generated samples as real samples. The generator uses Wasserstein loss as its loss function and incorporates feedback from the discriminator to continuously adjust the weights to generate higher-quality audio feature maps. For real sample xx and generated sample G(z), the loss function is: Formula 1: L D = E x∼pdata [D(x)] − E z∼pz [D(G(z))] Where D(x) represents the score of the discriminator for the real sample, and D(G(z)) represents the score for the generated sample.

4. The method for generating audio deep learning training data based on an adversarial neural network as described in claim 3, characterized in that, The loss function design in step 3 includes loss function design and improvement: the loss functions of the generator and discriminator are designed as adversarial losses, where the generator's loss is to maximize the probability that the generated sample is judged as a real sample by the discriminator, while the discriminator's loss is to improve the distinction between real and generated samples; to stabilize the training process, a feature matching loss is introduced into the generator; this loss requires the generator to not only generate samples that the discriminator cannot distinguish, but also generate samples that are more similar to real samples in terms of features; gradient penalty is added to Wasserstein GAN to prevent the discriminator from overfitting and improve the stability of the model.

5. The method for generating deep learning training data for audio based on adversarial neural networks as described in claim 4, wherein step 3 further includes model evaluation and optimization: after each round of training, signal feature evaluation, including signal-to-noise ratio, feature similarity, and human hearing tests, is used to evaluate the quality of the generated samples and select model weights with better generation performance; based on the model's loss convergence, the learning rates of the generator and discriminator are adaptively adjusted to accelerate model training and improve the quality of the generated audio.

6. The method for generating audio deep learning training data based on an adversarial neural network as described in claim 5, wherein the optimization step in step 3, which involves jointly training the generator and discriminator, includes the following optimization steps: Sampling real samples: from the real data distribution p data A batch of real samples x were sampled from the middle; Sample generation: from noise distribution p z The noise z is sampled and then generated into a sample G(z) by a generator G; Calculate the loss: Use the Wasserstein loss function L described above. D Calculate the loss of the discriminator; Backpropagation: The gradient of the loss with respect to the discriminator parameters is calculated using the backpropagation algorithm; Update weights: Update the discriminator weights using the Adam optimization algorithm; Weight pruning: After updating the weights, ensure that the discriminator's weights are within the preset range; The generator maximizes the output of the discriminator: L G =−E z∼pz [D(G(z))] That is, the generator hopes that the generated samples will get higher scores, so as to approximate the distribution of the real samples; Termination condition for joint training of generator and discriminator: The training process terminates when one of the following conditions is met:

1. The generated audio samples meet the preset quality standards in signal characteristics or auditory tests; 2. The loss functions of the generator and discriminator are basically stable over multiple cycles, indicating that the model has reached convergence.

Images