A pseudo-label filtering and correction method and system for self-supervised speaker recognition

Abstract

This application proposes a pseudo-label filtering and correction method and system for self-supervised speaker recognition. First, easy samples are determined based on the consistency between the model-predicted category and the pseudo-label category. Then, the class centers of each pseudo-label category are estimated using these easy samples. The confidence of the pseudo-label is measured by the similarity between the speech sample embedding representation and its corresponding class center. A binary probability model is used to dynamically divide the reliable and unreliable sample sets. Finally, label reassignment is performed on the unreliable samples based on the nearest class center. Based on the corrected similarity, the unreliable samples are further divided into corrected reliable samples, weighted learning samples, and rejection samples, and different samples are trained with different training weights. This application avoids the generalization problem caused by manually fixed thresholds and reduces the impact of erroneous pseudo-labels on parameter updates while utilizing ambiguous samples, thus synergistically improving the quality of pseudo-labels and the representational ability of the encoder.

Description

Technical Field

[0001] This application relates to the field of speaker recognition technology, specifically to a pseudo-label filtering and correction method and system for self-supervised speaker recognition. Background Technology

[0002] Speaker verification technology is used to determine whether two audio clips come from the same speaker. It is a fundamental technology in scenarios such as voiceprint recognition, identity authentication, smart cockpits, conference systems, customer service quality inspection, and voice security. In recent years, deep neural networks have significantly improved the performance of speaker verification systems, but mainstream supervised training methods usually rely on large-scale, high-quality speaker-annotated data. The high cost of audio data collection and manual annotation, as well as privacy concerns, limit the continuous iteration of speaker verification models across multiple scenarios and large-scale data.

[0003] Existing self-supervised speaker verification methods include encoder pre-training based on SimCLR, DINO, etc., and pseudo-label generation based on K-Means clustering; they also include sample filtering based on loss gating, dynamic decision thresholding and label correction based on loss GMM, and hierarchical training strategies for robust training of noisy labels. Their common drawbacks are: the quality of pseudo-labels is limited by the clustering algorithm and the preset number of clusters; loss-based confidence assessment is prone to misclassifying difficult samples as noisy samples; and filtered samples cannot be fully utilized, leading to decreased data utilization.

[0004] To reduce reliance on manually labeled data, self-supervised speaker verification methods typically employ a two-stage training process: the first stage trains a speaker encoder using unlabeled speech to extract stable speaker embeddings; the second stage uses this encoder to extract embedding vectors from all training speech, then generates pseudo-labels through clustering algorithms, and uses these pseudo-labels as supervisory signals to fine-tune the encoder in a classification-based manner. While existing techniques can fully utilize large amounts of unlabeled speech, their performance is highly dependent on the quality of the pseudo-labels.

[0005] Existing pseudo-label generation and filtering schemes still have significant problems. First, traditional K-Means clustering usually requires a pre-defined number of speakers or clusters and implicitly assumes that the data distribution has a convex structure, making it difficult to fully characterize the complex topological relationships in the high-dimensional speaker embedding space. When the actual number of speakers is unknown or the data contains noise, duplicate identities, or cross-domain audio, cluster pseudo-labels are prone to merging or splitting errors. Second, pseudo-label filtering methods based on classification loss usually assume that low-loss samples are more reliable and identify noisy samples through a fixed threshold. However, the loss value is affected not only by the correctness of the label but also by factors such as sample difficulty, audio quality, channel, accent, and duration. High-loss samples may contain a large number of "difficult samples" with correct labels but high discrimination difficulty. Directly filtering such samples will reduce data utilization and weaken the model's discrimination ability. Third, some methods directly discard samples deemed unreliable. Although this can reduce noise interference, it wastes a large number of boundary samples and long-tail speaker samples, which is not conducive to the model obtaining sufficient supervision from unlabeled data.

[0006] The existing patent "CN119207428A A Deep Learning Method for Filtering Noise Labels in Voiceprint Datasets" identifies reliable sample labels by matching sample labels with the model's top-k predictions, and then divides the dataset into reliable and unreliable sets. For the reliable set, the voiceprint model is trained using these data; for the unreliable set, strong and weak data augmentation are used to generate two views for each sample, then the consistency regularization loss is calculated, and backpropagation is performed. However, the top-k predictions in this patent do not intuitively reflect the reliability of the labels, and the fixed filtering threshold cannot adapt to different noise distributions.

[0007] Therefore, how to dynamically evaluate the credibility of pseudo-labels without manual annotation and reduce the reliance on fixed thresholds and loss distribution assumptions; how to distinguish between "real noise samples" and "difficult samples whose labels may be correct but are difficult to learn"; and how to improve the utilization rate of unlabeled data while suppressing the influence of noisy labels are the technical problems that existing technologies urgently need to solve. Summary of the Invention

[0008] This application addresses one or more technical deficiencies in the prior art by proposing the following technical solution.

[0009] Based on the first aspect of this application, a pseudo-label filtering and correction method for self-supervised speaker recognition is proposed, including: S1: The system predicts the category of the unlabeled speech training set through the speaker encoder and classifier, and identifies the speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; S2: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ; in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, where j represents the j-th easy sample; S3: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. S4: Based on the class center corresponding to the maximum similarity, the system corrects the pseudo-label category of the unreliable sample to the pseudo-label category corresponding to the class center of the maximum similarity, thus obtaining a new pseudo-label category; S5: The system calculates the class center of the new pseudo-label category and the second similarity with the corresponding embedding representation. Based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM component estimated by the EM algorithm, the unreliable samples are divided into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set. S6: The system uses the reliable sample set, the modified reliable sample set, and the weighted learning sample set as a joint sample set, and updates the speaker encoder and classifier parameters through the joint sample set.

[0010] This application identifies samples whose predicted category matches the pseudo-label category as "easy samples" and uses these easy samples to estimate the robust class centers of each pseudo-label category. This avoids the contamination of class centers by erroneous pseudo-labels, making the class centers more reflective of the true geometric centers of each category. By using class center similarity as a confidence index, this application avoids misclassifying difficult samples with correct labels but high discrimination difficulty as noise. The sample reuse strategy for unreliable samples can maximize data utilization while suppressing noise.

[0011] Furthermore, if the second similarity is greater than or equal to the dynamic decision threshold, then the unreliable sample is assigned to the corrected reliable sample set.

[0012] Furthermore, if the second similarity is greater than or equal to the mean of the minimum GMM components and the second similarity is less than the dynamic decision threshold, then unreliable samples are assigned to the weighted learning sample set.

[0013] Furthermore, the classification loss is weighted based on the second similarity, and the calculation formula is as follows: ; in, This represents the weighted loss function. This represents the i-th speech sample. This represents the pseudo-label after correction for speech sample i. Represents a classifier. Speaker encoder, This represents the second similarity calculated after correction of speech sample i. This represents the classification loss function.

[0014] This step utilizes fuzzy samples while suppressing the impact of potential erroneous pseudo-labels on parameter updates, thus achieving pseudo-label correction.

[0015] Furthermore, if the second similarity is less than the mean of the smallest GMM component, the unreliable sample is assigned to the rejection sample set.

[0016] This step involves selecting and filtering out the rejected sample set, which can prevent extreme noise from interfering with model training and ensure the quality of training data.

[0017] Furthermore, the embedding representation of the unlabeled speech training set is extracted by a pre-trained speaker encoder and L2 normalized. The embedding representation is then clustered to obtain a pseudo-label for each speech sample.

[0018] Furthermore, the Infomap community discovery algorithm is used for clustering.

[0019] The Infomap community discovery algorithm in this step does not require pre-specifying the number of clusters and can generate pseudo-labels by utilizing the topological structure between samples in a high-dimensional embedding space.

[0020] Furthermore, the binary probability model adopts a two-component Gaussian mixture model and estimates the mean and variance of the GMM components using the EM algorithm. A dynamic decision threshold is calculated based on the mean and variance of the GMM components. Speech samples with a similarity greater than or equal to the dynamic decision threshold are classified as reliable samples, while speech samples with a similarity less than the dynamic decision threshold are classified as unreliable samples.

[0021] This step fits the similarity set using a binary probability model and automatically learns a dynamic decision threshold. It eliminates the need for manually setting a fixed threshold and can adapt to changes in similarity distribution across different data scales, clustering qualities, and training stages.

[0022] Based on the second aspect of this application, a pseudo-label filtering and correction system for self-supervised speaker recognition is also proposed, comprising: Easy Sample Recognition Module: The system uses the speaker encoder and classifier to predict the category of the unlabeled speech training set, and identifies speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; Class center module: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ; in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, and j represent the sample index of the easy sample. Sample segmentation module: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. Pseudo-label correction module: Based on the class center corresponding to the maximum similarity, the system corrects the pseudo-label category of unreliable samples to the pseudo-label category corresponding to the class center of the maximum similarity, thus obtaining a new pseudo-label category; Re-partitioning module: The system calculates the second similarity between the class center of the new pseudo-label category and the corresponding embedding representation, and divides the unreliable samples into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM components estimated by the EM algorithm; Parameter update module: The system uses the reliable sample set, the corrected reliable sample set, and the weighted learning sample set as a joint sample set to update the speaker encoder and classifier parameters.

[0023] Based on a third aspect of this application, a computer program product is also proposed, on which one or more computer programs are stored, which, when executed by a computer processor, implement the method described above.

[0024] The technical advantages of this application are as follows: This application proposes a pseudo-label filtering and correction method and system for self-supervised speaker recognition. It uses class center similarity instead of simple loss value as the pseudo-label confidence index, which can reduce the interference of sample difficulty on confidence judgment; it estimates robust class centers based on easy samples, reducing the contamination of class centers by erroneous pseudo-labels; it then uses the similarity between speech samples and class centers to dynamically filter pseudo-labels, making the filtering process closer to the geometric structure of the speaker embedding space; it automatically learns the similarity threshold through a Gaussian mixture model, eliminating the need for manually setting a fixed threshold, and can adapt to similarity changes under different data scales, different clustering qualities, different training stages, and different speaker distributions; it can recover some valuable data from samples that were originally judged as low-confidence through recent class center reassignment; and it can suppress potential noise while utilizing data by weighted learning of samples with medium confidence. This application can achieve a balance between data utilization and noise suppression, improve the quality of encoder representation and pseudo-labels, and enable the model to converge quickly with a small number of iterations. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart of a pseudo-label filtering and correction method for self-supervised speaker recognition provided according to an embodiment of this application.

[0027] Figure 2 This is a block diagram of a pseudo-label filtering and correction system for self-supervised speaker recognition provided according to an embodiment of this application.

[0028] Figure 3 This is a flowchart illustrating the specific implementation steps of a pseudo-label filtering and correction system for self-supervised speaker recognition, provided according to an embodiment of this application.

[0029] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present application. Detailed Implementation

[0030] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] Figure 1 This application illustrates a pseudo-label filtering and correction method for self-supervised speaker recognition, comprising: S1: The system predicts the category of the unlabeled speech training set through the speaker encoder and classifier, and identifies the speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; S2: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ; in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, where j represents the j-th easy sample; S3: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. S4: Based on the class center corresponding to the maximum similarity, the system corrects the pseudo-label category of the unreliable sample to the pseudo-label category corresponding to the class center of the maximum similarity, thus obtaining a new pseudo-label category; S5: The system calculates the class center of the new pseudo-label category and the second similarity with the corresponding embedding representation. Based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM component estimated by the EM algorithm, the unreliable samples are divided into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set. S6: The system uses the reliable sample set, the modified reliable sample set, and the weighted learning sample set as a joint sample set, and updates the speaker encoder and classifier parameters through the joint sample set.

[0033] It should be noted that this application identifies samples whose predicted category matches the pseudo-label category as "easy samples" and uses easy samples to estimate the robust class center of each pseudo-label category, thus avoiding the contamination of the class center by erroneous pseudo-labels and making the class center more reflective of the true geometric center of each category. The application uses class center similarity as a confidence index to avoid misclassifying difficult samples with correct labels but high discrimination difficulty as noise. The sample reuse strategy for unreliable samples can maximize the utilization of data while suppressing noise.

[0034] It should be noted that if the second similarity is greater than or equal to the dynamic decision threshold, then the unreliable sample will be assigned to the corrected reliable sample set.

[0035] It should be noted that if the second similarity is greater than or equal to the mean of the smallest GMM component and the second similarity is less than the dynamic decision threshold, then unreliable samples will be assigned to the weighted learning sample set.

[0036] It should be noted that the classification loss is calculated using a weighted average based on the second similarity score, and the calculation formula is as follows: ; in, This represents the weighted loss function. This represents the i-th speech sample. This represents the pseudo-label after correction for speech sample i. Represents a classifier. Speaker encoder, This represents the second similarity calculated after correction of speech sample i. This represents the classification loss function.

[0037] It should be noted that this step, while utilizing fuzzy samples, can suppress the impact of potential erroneous pseudo-labels on parameter updates, thereby achieving pseudo-label correction and reducing the impact of potential erroneous pseudo-labels.

[0038] It should be noted that if the second similarity is less than the mean of the smallest GMM component, the unreliable sample will be assigned to the rejection sample set.

[0039] It should be noted that this step involves screening out and filtering out the rejection sample set to avoid interference from extreme noise in model training and to ensure the quality of the training data.

[0040] It should be noted that the embedding representation of the unlabeled speech training set is extracted by the pre-trained speaker encoder and L2 normalized. The embedding representation is then clustered to obtain the pseudo-label of each speech sample.

[0041] It should be noted that the Infomap community discovery algorithm was used for clustering.

[0042] It should be noted that the Infomap community discovery algorithm in this step does not require pre-specifying the number of clusters and can generate pseudo-labels by utilizing the topological structure between samples in a high-dimensional embedding space.

[0043] It should be noted that the binary probability model adopts a two-component Gaussian mixture model, and estimates the mean and variance of the GMM components through the EM algorithm. A dynamic decision threshold is calculated based on the mean and variance of the GMM components. Speech samples with similarity greater than or equal to the dynamic decision threshold are classified as reliable samples, and speech samples with similarity less than the dynamic decision threshold are classified as unreliable samples.

[0044] It should be noted that this step fits the similarity set using a binary probability model and automatically learns the dynamic decision threshold. There is no need to manually set a fixed threshold, and it can adapt to changes in similarity distribution with different data scales, different clustering qualities, and different training stages.

[0045] It should be noted that this application selects easy samples by matching model prediction with clustering pseudo-label consistency, and uses easy samples instead of all samples for class center estimation, which can improve the robustness of class centers; it measures the reliability of pseudo-labels by the similarity between the embedded representation of the speech sample and the pseudo-label class centers, which is less likely to mistakenly delete difficult samples than filtering by simply using loss values; it adopts a two-component Gaussian mixture model, which can automatically learn the boundary threshold between reliable and unreliable samples, avoiding the generalization problem caused by manually fixing the threshold; it reassigns low-confidence samples to the category corresponding to the most similar class center and further divides unreliable samples according to the corrected similarity, which can reduce the impact of erroneous pseudo-labels on model parameter updates; and it re-extracts the embedded representation and re-clusters and performs filtering and correction after each round of training, so that the quality of pseudo-labels and the representation ability of encoder are synergistically improved.

[0046] The following is for reference. Figure 2 It illustrates a pseudo-label filtering and correction system for self-supervised speaker recognition, comprising: Easy sample recognition module 201: The system uses the speaker encoder and classifier to predict the category of the unlabeled speech training set, and recognizes the speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; Class center module 202: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ; in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, and j represent the sample index of the easy sample. Sample segmentation module 203: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. Pseudo-label correction module 204: Based on the class center corresponding to the maximum similarity value, the system corrects the pseudo-label category of the unreliable sample to the pseudo-label category corresponding to the class center corresponding to the maximum similarity value, thus obtaining a new pseudo-label category; Re-partitioning module 205: The system calculates the second similarity between the class center of the new pseudo-label category and the corresponding embedding representation, and divides the unreliable samples into a corrected reliable sample set, a weighted learning sample set and a rejection sample set according to the second similarity, the dynamic decision threshold and the mean of the minimum GMM component estimated by the EM algorithm; Parameter update module 206: The system uses the reliable sample set, the corrected reliable sample set, and the weighted learning sample set as a joint sample set, and updates the speaker encoder and classifier parameters through the joint sample set.

[0047] It should be noted that, as Figure 3 As shown, in STAGE I, the speaker encoder is trained using an unlabeled speech training set through self-distillation; in STAGE II, the embedded representation of the speech training set is extracted using the trained speaker recognition encoder, and pseudo-labels are generated through a clustering algorithm and used as supervision signals to fine-tune the speaker encoder. Specifically, it includes: The first step is to generate initial pseudo-tags; Speaker encoder obtained from pre-training or the previous iteration Extracting unlabeled speech training set The embedded representation is then subjected to L2 normalization: ; Preferably, Infomap is used to embed the representation set. Clustering is performed to obtain pseudo-labels for each speech segment. This constitutes the speech training set. ; The second step is to identify easy samples; In each training round, the current encoder is used and classifier The model predicts the category of the speech sample. Clustering pseudo-labels If the samples are consistent, then the speech sample is considered an easy sample; ; in, This represents the pseudo-labels predicted by the model. This indicates an embedded representation. Represents a classifier; The third step is to estimate the class centers based on easy samples; Calculate the normalized class center Oc using the embedding representation of each pseudo-label category c in the easy sample set: ; in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, and j represent the sample index of the easy sample. The fourth step is to calculate class center similarity and dynamically filter out pseudo-labels; For any sample i in the speech training set, compute its embedding representation. Instead of pseudo-labels Corresponding class center Similarity between (Cosine similarity is preferred): ; in, An embedding representation of a speech sample. Indicates similarity. Indicates pseudo-tags Corresponding class center; To avoid manually setting fixed thresholds, a binary probability model is used to fit the similarity set. A two-component Gaussian mixture model (GMM) is used, and the mean of the two GMM components is estimated using the EM algorithm. , and dynamic decision threshold Components with similarity greater than or equal to the dynamic decision threshold are classified as reliable components, while components with similarity less than the dynamic decision threshold are classified as unreliable samples, thus obtaining the reliable sample set. With unreliable sample sets : ; ; Fifth step: Correct low-confidence pseudo-labels; For unreliable sample sets Each speech sample is not discarded directly, but rather pseudo-labels are reassigned using the similarity between the speech sample and all class centers; specifically, the speech samples are... The new pseudo-tag is set to the category corresponding to the class center that is most similar to its embedded representation: ; Calculate the second similarity score between the new pseudo-label class center and its corresponding embedding representation: ; in, This represents the corrected second similarity. This indicates an embedded representation. Indicates the class center corresponding to the new pseudo-tag. Indicates the new pseudo-tag category; make According to the second similarity unreliable sample sets It is further divided into three subsets, specifically: like If the speech sample has a high degree of matching with the new category center after label correction, then the unreliable sample is added to the corrected reliable sample set. and use new pseudo tags Participate in training; like If the sample still has some uncertainty, but is not completely invalid, then the unreliable sample is added to the weighted learning sample set. This value is retained for training, and the loss is weighted according to the second similarity to reduce the impact of potential false labels. The weighted loss calculation formula is as follows: ; in, This represents the weighted loss function. This represents the i-th speech sample. This represents the pseudo-label after correction for speech sample i. Represents a classifier. Speaker encoder, This represents the second similarity calculated after correction of speech sample i. The classification loss function can be represented by weights, which can also be normalized similarity, GMM posterior probability, temperature-scaled Softmax probability, confidence squared, piecewise linear function or Focal weights. like If a speech sample does not match any available class center, it is considered to be either strong noise, abnormal speech, extremely short speech, speech from a non-target speaker, or a clustering error sample. It is then removed from the current training round and reassigned to the rejection sample set. ; The sixth step involves the final training objective being a joint training set comprised of the original reliable sample set, the revised reliable sample set, and the weighted learning sample set. The model then uses this joint training set... Update encoder and classifier parameters: ; After each round of training, select the encoder with the lowest EER on the validation set or the best training stability to re-extract all sample embeddings, and perform clustering, pseudo-label filtering, pseudo-label correction and classification training again. When entering a new iteration, the encoder parameters from the previous iteration are inherited and the classification head is reinitialized to reduce overfitting of the classification head to the old pseudo-label space. The iteration terminates when a fixed number of iterations is reached, the performance of the validation set no longer improves, the rate of change of cluster labels is lower than the threshold, or the similarity distribution tends to stabilize.

[0048] It should be noted that similarity Similarity is used to characterize the degree of matching between sample i and the current pseudo-label category. The higher the similarity, the closer the sample is to the robust class center of its pseudo-label category, and the more reliable its pseudo-label is; The lower the value, the more likely the sample is a cross-cluster error, an outlier, or a high-noise sample.

[0049] In a specific implementation, when there are no easy samples for the pseudo-label category at the current stage, the normalized mean of all samples of the category, the class center saved in the previous iteration, the exponential moving average class center, or the category is temporarily marked as a low-confidence category and its weight is reduced in subsequent training.

[0050] It should be noted that easy samples typically have high intra-class consistency and low label noise probability, making them suitable for estimating stable class centers. Compared to directly using all samples to calculate class centers, using easy samples to calculate class centers can reduce the contamination of class centers by erroneous pseudo-labels.

[0051] It should be noted that this application uses class center similarity as a pseudo-label confidence index, which can reduce the interference of sample difficulty on confidence judgment compared with the simple loss value index. This application does not discard difficult samples with high loss but close to the correct class center. Instead, it reassigns unreliable samples through the nearest class center, which can recover some valuable data from samples that were originally judged as low confidence. Weighted learning is performed on samples with medium confidence, which can suppress potential noise while utilizing data, improve data utilization, and retain training samples that are valuable for discrimination boundary.

[0052] It should be noted that the robust class center estimation based on easy samples can reduce the contamination of class centers by erroneous pseudo-labels; and the dynamic filtering of pseudo-labels by the similarity between speech samples and class centers can make the filtering process closer to the geometric structure of the speaker embedding space.

[0053] It should be noted that this application automatically learns the similarity threshold through a Gaussian mixture model, eliminating the need for manually setting a fixed threshold. This allows it to adapt to similarity variations under different data scales, clustering qualities, training stages, and speaker distributions.

[0054] It should be noted that the progressive training strategy of "filtering-correction-weighting-iteration" in this application can achieve a balance between noise suppression and data utilization, and can gradually improve the quality of pseudo-labels as the quality of encoder representation improves, enabling the model to converge quickly within a small number of iterations. This application can further reduce the equal error rate (EER) on the VoxCeleb1 test set compared to training with only Infomap pseudo-labels. After pseudo-label correction, the Vox-O EER in the first iteration was reduced to about 1.048%, and in the second iteration it further reached about 0.95%, which verifies the effectiveness of this application in self-supervised speaker verification application scenarios.

[0055] It should be noted that this application can combine SDPN, DINO, ECAPA-TDNN, Infomap, and AAM-Softmax, and can also be transferred to other unsupervised or semi-supervised pseudo-label learning tasks.

[0056] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0057] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage section 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for system operation. CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0058] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a liquid crystal display (LCD) and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card and a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage section 408 as needed.

[0059] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0060] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0062] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0063] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: perform category prediction on an unlabeled speech training set using a speaker encoder and classifier, and identify speech samples whose predicted categories match pre-obtained pseudo-label categories as easy samples; calculate the normalized class center corresponding to each pseudo-label category based on the embedding representation of the speech samples in the speech training set and the easy samples; calculate the similarity between the embedding representation of each speech sample and the class center, fit the similarity set using a binary probability model, and classify the speech samples according to the dynamic decision threshold estimated by the EM algorithm. The system categorizes samples into reliable and unreliable samples. Based on the class center corresponding to the maximum similarity value, the system corrects the pseudo-label category of the unreliable sample to the pseudo-label category corresponding to the class center corresponding to the maximum similarity value, thus obtaining a new pseudo-label. The system calculates the second similarity between the class center of the new pseudo-label category and the corresponding embedding representation. Based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM components estimated by the EM algorithm, the system divides the unreliable samples into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set. The system uses the reliable sample set, the corrected reliable sample set, and the weighted learning sample set as a joint sample set to update the speaker encoder and classifier parameters.

[0064] Finally, it should be noted that the above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A pseudo-label filtering and correction method for self-supervised speaker recognition, characterized in that, include: S1: The system predicts the category of the unlabeled speech training set through the speaker encoder and classifier, and identifies the speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; S2: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ； in, 'c' represents the class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, and j represent the sample index of the easy sample. S3: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. S4: Based on the class center corresponding to the maximum similarity, the system corrects the pseudo-label category of unreliable samples to the pseudo-label category corresponding to the class center corresponding to the maximum similarity, thus obtaining a new pseudo-label category; S5: The system calculates the class center of the new pseudo-label category and the second similarity with the corresponding embedding representation. Based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM component estimated by the EM algorithm, the unreliable samples are divided into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set. S6: The system uses the reliable sample set, the modified reliable sample set, and the weighted learning sample set as a joint sample set, and updates the speaker encoder and classifier parameters through the joint sample set.

2. The method according to claim 1, characterized in that, If the second similarity is greater than or equal to the dynamic decision threshold, then the unreliable sample is assigned to the corrected reliable sample set.

3. The method according to claim 1, characterized in that, If the second similarity is greater than or equal to the mean of the minimum GMM components and the second similarity is less than the dynamic decision threshold, then the unreliable sample is assigned to the weighted learning sample set.

4. The method according to claim 3, characterized in that, The classification loss is weighted based on the second similarity, and the calculation formula is as follows: ； in, This represents the weighted loss function. This represents the i-th speech sample. This represents the pseudo-label after correction for speech sample i. Represents a classifier. Speaker encoder, This represents the second similarity calculated after correction of speech sample i. This represents the classification loss function.

5. The method according to claim 1, characterized in that, If the second similarity is less than the mean of the smallest GMM component, then the unreliable sample is assigned to the rejection sample set.

6. The method according to claim 1, characterized in that, The embedded representations of the unlabeled speech training set are extracted by a pre-trained speaker encoder and L2 normalized. The embedded representations are then clustered to obtain pseudo-labels for each speech sample.

7. The method according to claim 6, characterized in that, Clustering was performed using the Infomap community discovery algorithm.

8. The method according to claim 1, characterized in that, The binary probability model adopts a two-component Gaussian mixture model and estimates the mean and variance of the GMM components using the EM algorithm. A dynamic decision threshold is calculated based on the mean and variance of the GMM components. Speech samples with a similarity greater than or equal to the dynamic decision threshold are classified as reliable samples, and speech samples with a similarity less than the dynamic decision threshold are classified as unreliable samples.

9. A pseudo-label filtering and correction system for self-supervised speaker recognition, characterized in that, include: Easy Sample Recognition Module: The system uses a speaker encoder and classifier to predict the category of the unlabeled speech training set, and identifies speech samples whose predicted category matches the pre-obtained pseudo-label category as easy samples; Class center module: Based on the embedded representation of speech samples in the speech training set and the easy samples, the system calculates the normalized class center corresponding to each pseudo-label category. The calculation formula is as follows: ； in, 'c' represents the normalized class center, and 'c' represents the pseudo-label category. Let i represent the embedding representation of the speech sample, where i represents the i-th speech sample. Indicates a pseudo-tag; Let j represent the set of easy samples, and j represent the sample index of the easy sample. Sample segmentation module: The system calculates the similarity between the embedded representation of each speech sample and the class center, fits the similarity set through a binary probability model, and divides the speech samples into reliable samples and unreliable samples according to the dynamic decision threshold estimated by the EM algorithm. Pseudo-label correction module: Based on the class center corresponding to the maximum similarity, the system corrects the pseudo-label category of unreliable samples to the pseudo-label category corresponding to the class center corresponding to the maximum similarity, thus obtaining a new pseudo-label category; Re-partitioning module: The system calculates the second similarity between the class center of the new pseudo-label category and the corresponding embedding representation, and divides the unreliable samples into a corrected reliable sample set, a weighted learning sample set, and a rejection sample set based on the second similarity, the dynamic decision threshold, and the mean of the minimum GMM components estimated by the EM algorithm; Parameter update module: The system uses the reliable sample set, the corrected reliable sample set, and the weighted learning sample set as a joint sample set to update the speaker encoder and classifier parameters.

10. A computer program product having one or more computer programs thereon, characterized in that, When the computer program is executed by a computer processor, the method described in any one of claims 1-8 is performed.

Images