Deep learning method for filtering noise labels in voiceprint data set

Abstract

The invention relates to a deep learning method for filtering noise tags in a voiceprint data set, and aims to improve the robustness and accuracy of a voiceprint recognition model for noise tag data. According to the method, a two-stage training method is adopted, the first stage is a warm-up stage, and a voiceprint recognition model is trained for a few training periods by using all data, so that the model obtains a certain ability to recognize a speaker; in the process, a reliable sample label is marked by matching the sample label with the model top-k prediction. In the second stage, the data set is divided into a reliable data set and an unreliable data set according to whether a sample has a reliable label or not; for a reliable data set, training a voiceprint model by using the data; for an unreliable data set, two views are generated for each sample by adopting strong data augmentation and weak data augmentation, then consistency regularization loss is calculated, and back propagation is carried out. And meanwhile, for each sample in the unreliable data set, inputting the sample into the network for prediction, judging whether a sample label is matched with a top-k prediction value or not, and if the sample label is matched with the top-k prediction value, transferring the sample from the unreliable data set to a reliable data set. And through repeated iterative training, a trained voiceprint recognition model and a data set subjected to reliability screening are finally obtained. The method can improve the robustness and accuracy of the voiceprint recognition model for noise label data, and has a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the field of voiceprint recognition, and mainly relates to a deep learning method for filtering noise labels in a voiceprint data set. Background Art

[0002] Voiceprint recognition is a technology that uses voice signals to identify people. It uses the characteristics of a speaker's voice to identify them, just like fingerprint recognition and facial recognition. Voiceprint recognition technology has a wide range of applications in security, finance, and justice.

[0003] In voiceprint recognition, the quality of the dataset is crucial to model performance. However, in real-world applications, voiceprint datasets often have problems with noise and poor labels, which may affect the accuracy and reliability of voiceprint recognition. Traditional methods usually rely on manual filtering or manual cleaning of datasets to remove samples with poor labels or noise. This method requires a lot of time and manpower costs, and may weaken the diversity and representativeness of the dataset. Therefore, an automated method is needed to solve this problem. However, traditional deep learning methods usually suffer from overfitting or underfitting problems when dealing with voiceprint datasets with noise and poor labels, resulting in a decrease in model accuracy.

[0004] In order to solve this problem, the present invention proposes a new deep learning method, which filters the noise labels through a two-stage learning method, and uses the noise label data for consistency regularization to further improve the performance. Our method can automatically distinguish between noise labels and non-noise labels, thereby avoiding the problem of overfitting noise data and extracting wrong voiceprint features in traditional methods. The purpose of the present invention is to be able to train a voiceprint recognition model with excellent performance through our method even when there are noise labels in the voiceprint dataset. Compared with traditional methods, our method does not require manual filtering or manual cleaning of the dataset, which greatly reduces time and labor costs, while improving the accuracy and reliability of voiceprint recognition. Summary of the invention

[0005] The present invention provides a deep learning method for filtering noise labels in voiceprint datasets. In actual scenarios, due to label errors or noise interference, there may be a certain degree of noise labels in the training data of the voiceprint recognition model. These noise labels will affect the accuracy and reliability of the model, so how to improve the robustness of the model to noise labels is an important issue.

[0006] In this invention, we propose a two-stage training method. In the first stage, the warm-up phase, the voiceprint recognition model is trained using the complete voiceprint data set D, in order to enable the model to acquire a certain ability to identify the speaker. During the training process of this stage, for each sample, we match the sample label with the model prediction. If the sample label is included in the top k results of the predicted value, the sample is marked as a reliable label.

[0007] In the second stage of semi-supervised learning, the data set is divided into a reliable data set D1 and an unreliable data set D2 according to whether there are reliable labels. For the reliable data set D1, we use these data to train the voiceprint model to improve the accuracy of the model. For the unreliable data set D2, we perform strong data augmentation and weak data augmentation on each sample in it to obtain two views. Then these two views are input into the network to obtain two prediction values, and the consistency regularization loss is calculated for these two prediction values, and then back propagation is performed. This can improve the robustness of the model for the voiceprint recognition model. In addition, for each sample in the unreliable data set D2, we also pass it through the network to obtain a prediction, and then determine whether the sample label and the top-k prediction value match. If they match, the sample is moved from the unreliable data set D2 to the reliable data set D1. Through this screening process, the accuracy and robustness of the model for reliable labels can be further improved. Then repeat the second stage process until the end of training.

[0008] Finally, the voiceprint model training of the present invention will obtain a voiceprint recognition model and a screened reliable and unreliable data set. The method can effectively improve the robustness of the voiceprint recognition model to noise labels, improve the accuracy and reliability of the model, and is suitable for various speaker recognition scenarios, with broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is the framework of the method proposed by the present invention to filter noise labels in the voiceprint dataset

[0010] Figure 2 yes Figure 1 The process of matching sample labels and model predictions DETAILED DESCRIPTION

[0011] The present invention proposes a deep learning method for filtering noise labels in a voiceprint dataset, which is further described in detail below in conjunction with the accompanying drawings and specific implementation methods:

[0012] 1. The robust speaker model training method for noisy labels is as follows Figure 1 As shown, the following steps are included:

[0013] (1) The training process is defined as: setting the model to training mode, randomly selecting small batches of data from the data set, inputting them into the network to obtain output values, normalizing the output values ​​through softmax to obtain the probability p of each class, and then calculating the loss value by using the loss function to compare p with the corresponding label y. The loss function can use the common cross entropy loss, AM-Softmax or AAM-Softmax. After obtaining the loss value, back propagation is performed through the optimizer to update the network parameters, and small batches of data are repeatedly selected from the remaining data to perform this process until all data have been trained, and a training cycle ends. Strong data augmentation is defined as: making a large transformation to the original speech signal to generate new data that is significantly different from the original data, such as noise injection, spectrum perturbation, reverberation simulation, etc. Weak data augmentation is defined as: making a slight transformation to each speech signal to generate new data that is slightly different from the original data, such as volume adjustment, random truncation, etc.

[0014] (2) Obtaining the Dataset where x i is the i-th speech file in the speaker dataset, y i is the speaker id (label) of the i-th speech, and y i It could be a ground truth label (true label) or a noisy label (wrong label).

[0015] (3) Create data x i to label y i The deep learning model f(·;θ):x i →[0,1] c Where c represents the number of speakers in the training dataset, and the models that can be used by f(·; θ) include but are not limited to mainstream speaker recognition models such as TDNN, ResNet, ECAPA-TDNN, and MFA-Conformer.

[0016] (4) According to the training process in (1), Figure 1 Shown in the upper part. Using all data Train the speaker model f(·; θ). During the training process, for each training sample x i , the model's prediction value prediction for the sample i With the sample label y i by Figure 2 If the match is successful, the sample is marked as a reliable sample, otherwise no marking is performed. This method is used for about 4 to 10 training cycles.

[0017] (5) After the training process of (4) is completed, the data set If the sample is marked as a reliable sample, it is divided into the reliable data set Otherwise, it is divided into unreliable data sets middle.

[0018] (6) For reliable data sets The training is performed according to the training process in (1). Refer to the data augmentation method in (1) to perform strong data augmentation and weak data augmentation on each sample to obtain two views, and then input these two views into the voiceprint recognition model to obtain two prediction results. Then, the consistency regularization loss L is calculated for these two prediction results. consistency =Ex[d(f θ (T1(x)),f θ (T2(x)))], then L consistency The network parameters θ are updated by gradient back propagation. In addition to consistency regularization, each sample in D2 is input into the network to obtain a predicted value, and then the predicted value and the corresponding sample label are Figure 2 The label is matched to determine whether it is reliable. If it is a reliable label, the sample is moved from D2 to D1. The above process is repeated until the training is completed.

[0019] (7) When the training is finished, D2 is the filtered noise label data, and D1 is the retained clean label data. f(·; θ) is the trained speaker model, which can be used for speaker recognition and speaker verification tasks.

[0020] 2. How to judge whether the sample label is reliable? Figure 2 The detailed steps are as follows:

[0021] (1) Sort the predicted values ​​of the voiceprint model for the sample from large to small according to the probability value to obtain the corresponding probability value sequence and serial number sequence, where the serial number sequence corresponds one-to-one to the probability value sequence, and the serial number represents the speaker category.

[0022] (2) Select the first k sequence numbers as set P.

[0023] (3) Determine whether the sample label y is in the set P. If so, mark the label as a reliable label.

Claims

1. A deep learning method for filtering noise labels in a voiceprint dataset. The first stage is a warm-up stage, which includes the following steps: a. Obtain all training data of the voiceprint recognition model; b. Use all the data to train the voiceprint recognition model for a few training cycles, so that the model can acquire a certain ability to identify the speaker; c. In each training cycle, for each sample, match the sample label with the model prediction. If the top k results of the predicted value contain the sample label, the sample label is marked as a reliable label.

2. The second stage is the semi-supervised learning stage, which includes the following steps: a. Divide the data set into reliable data sets and unreliable data sets based on whether there are reliable labels; b. For reliable data sets, use these data to train the voiceprint model to improve the accuracy of the model; c. For unreliable data sets, perform strong data augmentation and weak data augmentation on each sample to obtain two views, then input the two views into the network to obtain two prediction values, calculate the consistency regularization loss for the two prediction values, and then perform back propagation; d. For each sample in the unreliable data set, input it into the network to obtain the predicted value, and then determine whether the sample label and the top-k predicted value match. If they match, move the sample from the unreliable data set to the reliable data set; e. Repeat steps b to d until the training is completed. After the voiceprint recognition model training is completed, a trained voiceprint recognition model and a screened reliable and unreliable data set are obtained.

3. The speaker model training method that is robust to noisy labels according to claim 1, wherein in the first stage of training, the method for matching sample labels with model predictions is: matching sample labels with top-k prediction values ​​predicted by the model, if the sample label is among them, The sample is labeled as a reliable label.

4. According to the speaker model training method that is robust to noise labels as described in claim 1, the method of strong data augmentation in the second stage is: making a large transformation on the original speech signal to generate new data that is significantly different from the original data, such as noise injection, spectrum perturbation, reverberation simulation and other methods.

5. According to the speaker model training method that is robust to noise labels as described in claim 1, the method of weak data augmentation in the second stage is: slightly transform each speech signal to generate new data that is slightly different from the original data, such as volume adjustment, random truncation, etc.

6. According to the speaker model training method that is robust to noisy labels as described in claim 1, the consistency regularization loss of the second stage can be obtained by L1 / L2 norm, cosine similarity or KL divergence, and back-propagated.

7. According to the speaker model training method that is robust to noisy labels as described in claim 1, the method for determining whether the sample label and the top-k prediction value match in the second stage is: obtaining a prediction through the network, and then determining whether the sample label and the top-k prediction value match, and if they match, moving the sample from the unreliable data set to the reliable data set.

8. According to the speaker model training method that is robust to noise labels as described in claim 1, the method and number of strong data augmentation and weak data augmentation in the second stage can be adjusted according to specific circumstances.

9. According to the speaker model training method that is robust to noise labels as described in claim 1, wherein the voiceprint recognition model adopts a deep learning model, such as TDNN, ResNet, ECAPA-TDNN, Conformer, etc.

10. A voiceprint recognition system based on the speaker model training method that is robust to noise labels as described in claim 1, comprising a voiceprint recognition model and a training data set, wherein the training data set is screened to include a reliable data set and an unreliable data set.

Images