Speaker recognition model training method and device based on dynamic weighted mixed loss

Abstract

The invention provides a speaker recognition model training method and device based on dynamic weighted mixed loss. The method comprises the following steps: training a voiceprint feature extraction network by using a first loss function; freezing network parameters of the voiceprint feature extraction network, extracting an embedded vector of the training set sample based on the voiceprint feature extraction network, and training a generative PLDA network based on the embedded vector; the network parameters of the differentiable PLDA network are initialized based on the network parameters of the generative PLDA network, and the network parameters of the voiceprint feature extraction network are unfrozen; constructing a mixed loss function based on the dynamic smooth weight; and performing joint back propagation updating on the voiceprint feature extraction network and the differentiable PLDA network based on the mixed loss function to obtain a speaker recognition model. And an information feedback closed loop from the rear end to the front end is constructed, so that the final model can achieve better comprehensive performance on a plurality of key indexes, and the accuracy of speaker recognition is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speaker recognition, in particular to a speaker recognition model training method and device based on dynamic weighted hybrid loss. BACKGROUND

[0002] A typical speaker recognition system usually adopts a two-stage architecture of "front-end-back-end", however, this front-end-back-end separation design has a significant performance inconsistency problem. Although the PLDA back-end performs excellently in the EER index, in the Top-N accuracy of the speaker identification task (Identification), the simple cosine similarity scoring method is often superior to PLDA. The fundamental reason for this contradiction lies in the difference in the underlying mechanism of the two methods.

[0003] In order to solve the above problem, the prior art proposes some improvement schemes. For example, some researches propose "discriminative PLDA", which directly optimizes the PLDA parameters to maximize the score gap of same and different sample pairs, but this is easy to cause overfitting; some researches try to implement PLDA as a differentiable neural network layer (NPLDA) and jointly optimize it with the front-end network. However, this kind of joint optimization method usually adopts a "pre-training-fine-tuning" strategy, and completely replaces the original AAM-Softmax loss with a single PLDA related loss function in the fine-tuning stage. This "one-size-fits-all" approach has obvious defects: in the process of adapting the front-end features to the PLDA statistical assumption, it may cause the network to forget the fine angle discrimination information learned in the pre-training stage, causing the angle discrimination degree to degenerate, and ultimately failing to balance robustness and discriminativeness. In addition, these methods generally lack a dynamic mechanism and cannot adaptively adjust the optimization focus according to the difficulty of the input sample (such as the degree of channel mismatch). In summary, the prior art urgently needs a new method that can retain the strong discriminativeness of the front-end network and enhance the robustness of the back-end model during the training process of the speaker recognition model, and solve the "performance inversion" problem caused by static and single optimization strategy. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a speaker recognition model training method and device based on dynamic weighted hybrid loss, by introducing a "hybrid loss function" in the joint optimization stage, that is, retaining a first loss function based on angle margin and a second loss function based on differentiable PLDA, an information feedback closed loop from the back-end to the front-end is constructed, so that the final model can achieve better comprehensive performance in multiple key indicators, thereby improving the accuracy of speaker recognition.

[0005] The embodiment of the present application provides a speaker recognition model training method based on dynamic weighted hybrid loss, which comprises: The voiceprint feature extraction network is trained using a first loss function; wherein, the first loss function is a loss function based on angular margin. Freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of the training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors. The network parameters of the differentiable PLDA network are initialized based on the network parameters of the generative PLDA network, and the network parameters of the voiceprint feature extraction network are unfrozen. For each training batch, a dynamic smoothing weight is calculated, and a hybrid loss function consisting of the first loss function and the second loss function of the differentiable PLDA network is constructed based on the dynamic smoothing weight. The speaker recognition model is obtained by jointly backpropagating the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function, so as to enable speaker recognition based on the speaker recognition model.

[0006] In one possible implementation, calculating a dynamic smoothing weight for each training batch includes: Determine the average cosine similarity of all similar sample pairs within the current training batch; Determine the high-score average cosine similarity of all negative sample pairs in the current training batch whose average cosine similarity is greater than the preset threshold for negative samples; wherein, if there are no negative sample pairs in the current training batch that exceed the preset threshold for negative samples, then set the high-score average cosine similarity to the preset threshold for negative samples. Based on the mean cosine similarity, the high-score mean cosine similarity, and the preset performance target threshold, the optimization tendency score is determined. The optimization tendency score is processed based on a nonlinear mapping function to obtain instantaneous weights. Then, the instantaneous weights are processed by an exponential moving average to obtain the dynamic smooth weights of the current training batch.

[0007] In one possible implementation, the optimization tendency score is determined by the following formula: ; in, To optimize the propensity score, For high-scoring mean cosine similarity, The average cosine similarity, Set a threshold for negative samples. A preset threshold is set for positive samples. as well as These are positive weight hyperparameters.

[0008] In one possible implementation, the second loss function is determined through the following steps: The embedding vector pairs from the training batch are input into the differentiable PLDA network to determine the log-likelihood ratio score of the homology probability. The log-likelihood ratio score is mapped using the Sigmoid function to determine the probability value; Based on the probability value, the true label of the sample pair, and the binary cross-entropy formula, the second loss function is determined.

[0009] In one possible implementation, training batches are constructed in the following manner: Training batches are constructed based on PK sampling strategy, active data augmentation strategy and active data generation strategy to ensure that the batch contains a sufficient number of low-scoring positive samples and / or high-scoring negative samples.

[0010] In one possible implementation, the speaker recognition based on the speaker recognition model includes: Preprocessing and acoustic feature extraction are performed on the speech data to be recognized; The acoustic features are input into the speaker recognition model's voiceprint feature extraction network, and a query embedding vector is output. The query embedding vector is paired with one or more reference embedding vectors, and each pair of embedding vectors is input into the differentiable PLDA network of the speaker recognition model to determine a similarity score. Based on the similarity score, the system outputs whether the speech data to be recognized and the target speaker are the same person.

[0011] This application embodiment also provides a speaker recognition model training device based on dynamic weighted hybrid loss, the speaker recognition model training device comprising: The feature pre-training module is used to train the voiceprint feature extraction network using a first loss function; wherein, the first loss function is a loss function based on angular margin. The generative PLDA network training module is used to freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of the training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors. An initialization module is used to initialize the network parameters of the differentiable PLDA network based on the network parameters of the generative PLDA network, and to unfreeze the network parameters of the voiceprint feature extraction network. The construction module is used to calculate a dynamic smoothing weight for each training batch and construct a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and the second loss function of the differentiable PLDA network. The joint training module is used to perform joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model.

[0012] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the speaker recognition model training method based on dynamic weighted hybrid loss as described above are performed.

[0013] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the speaker recognition model training method based on dynamic weighted hybrid loss as described above.

[0014] This application provides a speaker recognition model training method and apparatus based on dynamic weighted hybrid loss. The speaker recognition model training method includes: training a voiceprint feature extraction network using a first loss function, wherein the first loss function is a loss function based on angular margin; freezing the network parameters of the voiceprint feature extraction network, extracting embedding vectors of training set samples based on the voiceprint feature extraction network, and training a generative PLDA network based on the embedding vectors; initializing the network parameters of a differentiable PLDA network based on the network parameters of the generative PLDA network, and unfreezing the network parameters of the voiceprint feature extraction network; calculating a dynamic smoothing weight for each training batch, constructing a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and a second loss function of the differentiable PLDA network; and performing joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model. By introducing a "hybrid loss function" in the joint optimization stage, which simultaneously retains the first loss function based on angle margin and the second loss function based on differentiable PLDA, an information feedback loop from the back end to the front end is constructed, enabling the final model to achieve better overall performance on multiple key indicators, thereby improving the accuracy of speaker recognition.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a speaker recognition model training method based on dynamic weighted hybrid loss, provided for an embodiment of this application; Figure 2 A flowchart of the speaker recognition process provided in the embodiments of this application; Figure 3 A schematic diagram of the structure of a speaker recognition model training device based on dynamic weighted hybrid loss provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0019] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of speaker recognition technology.

[0020] Research has revealed several improvements to existing technologies to address these issues. For example, some studies have proposed "discriminative PLDA," directly optimizing PLDA parameters to maximize the score difference between similar and dissimilar sample pairs, but this easily leads to overfitting. Other studies have attempted to implement PLDA as a differentiable neural network layer (NPLDA) and jointly optimize it with the front-end network. However, these joint optimization methods typically employ a "pre-training-fine-tuning" strategy, completely replacing the original AAM-Softmax loss with a single PLDA-related loss function during the fine-tuning phase. This "one-size-fits-all" approach has significant drawbacks: in adapting the front-end features to the PLDA statistical assumptions, the network may forget the fine-grained angle discrimination information learned during pre-training, resulting in a degradation in angle discrimination and ultimately failing to simultaneously achieve robustness and discriminativity. Furthermore, these methods generally lack a dynamic mechanism to adaptively adjust the optimization focus based on the difficulty of the input samples (such as the degree of channel mismatch). In summary, existing technologies urgently need a new method that can preserve the strong discriminative power of the front-end network and enhance the robustness of the back-end model during the training process of speaker recognition models, while solving the "performance inversion" problem caused by static, single optimization strategies.

[0021] Based on this, this application provides a speaker recognition model training method based on dynamic weighted hybrid loss. By introducing a "hybrid loss function" in the joint optimization stage, that is, simultaneously retaining the first loss function based on angle margin and the second loss function based on differentiable PLDA, an information feedback closed loop from the back end to the front end is constructed, so that the final model can achieve better comprehensive performance in multiple key indicators, thereby improving the accuracy of speaker recognition.

[0022] Please see Figure 1 , Figure 1 This is a flowchart illustrating a speaker recognition model training method based on dynamic weighted hybrid loss, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the speaker recognition model training method includes: S101: Train the voiceprint feature extraction network using the first loss function; wherein, the first loss function is a loss function based on angular margin.

[0023] In this step, the voiceprint feature extraction network is trained using the first loss function until training is complete.

[0024] Here, this step provides a stable feature space with good initial discriminative power for subsequent joint training. In practice, only a loss function based on angular residuals is used. For example, AAM-Softmax loss can be used to pre-train voiceprint feature extraction networks (such as ECAPA-TDNN). This loss function... The definition is as follows:

[0025] Where n represents the batch size. It is the current embedding vector. yes The corresponding real speaker's identity tag. This represents the embedding vector. Its corresponding table identity The angle between identity tags. Boundary penalty by Representation. Scaling factor It is applied to increase the range of the output log-likelihood. This loss function drives the network to learn an initial embedding space, achieving a high degree of angular clustering of embedding vectors for samples of the same class, and angular separation of embedding vectors for samples of different classes.

[0026] It should be noted that using a loss function based on angular margin to train the voiceprint feature extraction network is a conventional technique, and this will not be elaborated on further. S102: Freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of the training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors.

[0027] In this step, the network parameters of the voiceprint feature extraction network are frozen, the embedding vectors of the training set samples are extracted based on the voiceprint feature extraction network, and the generative PLDA network is trained based on the embedding vectors. Specifically, the network parameters of the voiceprint feature extraction network are frozen, and this network is used to extract embedding vectors from the entire training set. Based on these embedding vectors, a generative PLDA model is trained using the traditional expectation-maximization algorithm to obtain the prior parameters of the generative PLDA network. S103: Initialize the network parameters of the differentiable PLDA network based on the network parameters of the generative PLDA network, and unfreeze the network parameters of the voiceprint feature extraction network.

[0028] In this step, the network parameters of the differentiable PLDA network are initialized based on the network parameters of the generative PLDA network, and the network parameters of the voiceprint feature extraction network are unfrozen.

[0029] Here, a differentiable PLDA network is employed, which implements the log-likelihood ratio scoring function of generative PLDA as a differentiable computational graph. Subsequently, the parameters trained on the aforementioned generative PLDA network are used as initial weights and transferred to this differentiable PLDA network. Through this statistical initialization method of parameter transfer, the differentiable PLDA network possesses prior knowledge derived from the statistical distribution of real data before entering the subsequent joint optimization stage. This effectively solves the training difficulties and slow convergence problems that may be caused by random initialization, laying the foundation for achieving stable end-to-end joint optimization.

[0030] S104: For each training batch, calculate a dynamic smoothing weight, and construct a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and the second loss function of the differentiable PLDA network.

[0031] In this step, a dynamic smoothing weight is calculated for each training batch, and a hybrid loss function consisting of a first loss function and a second loss function of a differentiable PLDA network is constructed based on the dynamic smoothing weight.

[0032] In one possible implementation, calculating a dynamic smoothing weight for each training batch includes: A: Determine the average cosine similarity of all similar sample pairs within the current training batch.

[0033] Here, the average cosine similarity of all similar sample pairs within the current training batch is determined. .

[0034] B: Determine the high-score average cosine similarity of all negative sample pairs in the current training batch whose average cosine similarity is greater than the preset threshold for negative samples; wherein, if there are no negative sample pairs in the current training batch that exceed the preset threshold for negative samples, then set the high-score average cosine similarity to the preset threshold for negative samples.

[0035] Here, all samples exceeding the preset threshold for negative samples in the current batch are calculated. The average cosine similarity of negative sample pairs To ensure the robustness of the dynamic weighting mechanism, it is necessary to [address / compare] here. The calculation is processed for stability: if there is no cosine similarity exceeding 1 in the current batch. If the number of negative sample pairs, i.e., the number of difficult negative samples, is 0, then... The value is directly set to .

[0036] C: Based on the mean cosine similarity, the high-score mean cosine similarity, and the preset performance target threshold, the optimization tendency score is determined.

[0037] Here, the optimization tendency score is determined based on the mean cosine similarity, the high-score mean cosine similarity, and the preset performance target threshold.

[0038] In one possible implementation, the optimization tendency score is determined by the following formula: ; in, To optimize the propensity score, For high-scoring mean cosine similarity, The average cosine similarity, Set a threshold for negative samples. A preset threshold is set for positive samples. as well as These are positive weight hyperparameters.

[0039] Here, when hour, A positive term indicates that the driving model tends to... .when Below the preset threshold for positive samples hour, A positive term indicates that the driving model tends to... . The higher the score, the more it indicates a greater need. .

[0040] D: The optimization tendency score is processed based on a nonlinear mapping function to obtain instantaneous weights, and the instantaneous weights are processed by an exponential moving average to obtain the dynamic smooth weights of the current training batch.

[0041] Here, the instantaneous weights are obtained by processing the optimization tendency score according to the nonlinear mapping function, and the instantaneous weights are processed by exponential moving average to obtain the dynamic smooth weights of the current training batch.

[0042] Specifically, the optimization tendency score is mapped to the [0, 1] interval to serve as the instantaneous weight. ,use function: ; in, It is a scaling factor used to control the sensitivity of weight changes.

[0043] Here, to ensure training stability, an exponential moving average is used to dynamically smooth the weights. Smoothing. Assume... For the first The smoothing weights ultimately used in each training batch The smoothing coefficient is updated using the following algorithm:

[0044] In one possible implementation, the second loss function is determined through the following steps: (1): Input the embedding vector pairs in the training batch into the differentiable PLDA network to determine the log-likelihood ratio score of the homology probability.

[0045] Here, in the training batch, the construction 1 pair of embedding vectors Assign a real label to each pair. .if and If they belong to the same speaker (positive sample pair), then .if and If they belong to different speakers (negative sample pairs), then For each pair of vectors Input the differentiable PLDA module to determine the log-likelihood ratio score. The score Let be the log-likelihood ratio for "the pair of vectors comes from the same speaker".

[0046] (2): The log-likelihood ratio score is mapped using the Sigmoid function to determine the probability value.

[0047] Here, because It is a log-likelihood ratio, while the BCE loss requires a probability in the range [0,1]. Therefore, the sigmoid function is used to map it to a probability. : ; Based on the probability value, the true label of the sample pair, and the binary cross-entropy formula, the second loss function is determined.

[0048] Here, the second loss function for:

[0049] in, For the current training batch, when the sample pair is the same person ( When ), maximize When the sample pair is not from the same person Minimize This makes the scoring output of the PLDA module discriminative.

[0050] In one possible implementation, the hybrid loss function is determined by the following formula:

[0051] in, For a mixed loss function, For the first loss function, For the second loss function, For dynamic smoothing weights, For 1- .

[0052] Here, the weighting coefficient and These are not fixed hyperparameters, but rather dynamically and adaptively determined based on the characteristics of the samples in the current training batch. These two weights are designed following constraints, namely:

[0053] Therefore, the core of the entire dynamic mechanism lies in how to calculate the final smoothing weights. .

[0054] In one possible implementation, training batches are constructed by using a PK sampling strategy, an active data augmentation strategy, and an active data generation strategy to ensure that the batch contains a sufficient number of low-scoring positive samples and / or high-scoring negative samples.

[0055] Here, to ensure the effectiveness and stability of the dynamic weighting mechanism, this invention preferably employs a sampling strategy that ensures a sufficient number of difficult samples are included in each batch. This strategy may include one or more of the following steps: ① Basic sampling strategy: Each training batch is constructed using a PK sampling method, i.e., each batch contains P speakers, and K samples are randomly selected from each speaker (where...). ② Active data augmentation: To ensure that the training batches always contain enough low-scoring positive samples to drive... To address the loss, this invention can further employ an active data augmentation strategy. For example, data augmentation techniques such as adding noise, reverberation, and encoding / decoding can be used to generate perturbed versions of the original audio online or offline. When constructing training batches, original samples and perturbed samples can be directly paired and fed into the hybrid loss function. ③ Active data generation: To address the issue that high-scoring negative samples may be very sparse in real data, leading to... To address the instability of the loss-driven signal, this invention can further employ an active generation mechanism based on speech conversion. Specifically, a pre-trained, well-decoupled speech conversion model can be used to synthesize a "fake" speaker. To make its voiceprint characteristics match those of the real speaker They are very close in the embedding space. During the training phase, The real audio and the synthesized audio are used to construct a high-scoring negative sample pair, and then fed into... Training is performed within the loss function.

[0056] S105: Based on the hybrid loss function, perform joint backpropagation update on the voiceprint feature extraction network and the differentiable PLDA network to obtain the speaker recognition model, so as to perform speaker recognition based on the speaker recognition model.

[0057] In this step, the voiceprint feature extraction network and the differentiable PLDA network are jointly backpropagated and updated according to the hybrid loss function to obtain the speaker recognition model, so as to perform speaker recognition based on the speaker recognition model.

[0058] For further information, please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating the speaker recognition process provided in an embodiment of this application. Figure 2 As shown: S201: Preprocess the speech data to be recognized and extract acoustic features.

[0059] S202: Input the acoustic features into the speaker recognition model's voiceprint feature extraction network and output a query embedding vector.

[0060] In this step, the acoustic features are input into the speaker recognition model's voiceprint feature extraction network, and the query embedding vector is output.

[0061] S203: Pair the query embedding vector with one or more reference embedding vectors, and input each pair of embedding vectors into the differentiable PLDA network of the speaker recognition model to determine the similarity score. Based on the similarity score, output whether the speech data to be recognized and the target speaker are the same person.

[0062] In this step, the query embedding vector is paired with one or more reference embedding vectors, and each pair of embedding vectors is input into the differentiable PLDA network of the speaker recognition model to determine the similarity score. Based on the similarity score, the output is whether the speech data to be recognized and the target speaker are the same person.

[0063] It should be noted that the reference embedding vector is the pre-registered reference embedding vector of the target speaker.

[0064] Here, the similarity score is compared with a preset decision threshold. If the similarity score is greater than or equal to the decision threshold, the query voice and the target speaker are determined to be the same person; otherwise, they are determined to be different people.

[0065] This application provides a speaker recognition model training method based on dynamic weighted hybrid loss. The speaker recognition model training method includes: training a voiceprint feature extraction network using a first loss function, wherein the first loss function is a loss function based on angular margin; freezing the network parameters of the voiceprint feature extraction network, extracting embedding vectors of training set samples based on the voiceprint feature extraction network, and training a generative PLDA network based on the embedding vectors; initializing the network parameters of a differentiable PLDA network based on the network parameters of the generative PLDA network, and unfreezing the network parameters of the voiceprint feature extraction network; calculating a dynamic smoothing weight for each training batch, constructing a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and a second loss function of the differentiable PLDA network; and performing joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model. By introducing a "hybrid loss function" in the joint optimization stage, which simultaneously retains the first loss function based on angle margin and the second loss function based on differentiable PLDA, an information feedback loop from the back end to the front end is constructed, enabling the final model to achieve better overall performance on multiple key indicators, thereby improving the accuracy of speaker recognition.

[0066] Please see Figure 3 , Figure 3 This is a schematic diagram of a speaker recognition model training device based on dynamic weighted hybrid loss, provided as an embodiment of this application. Figure 3 As shown, the speaker recognition model training device 300 includes: The feature pre-training module 310 is used to train the voiceprint feature extraction network using a first loss function; wherein, the first loss function is a loss function based on angular margin. Generative PLDA network training module 320 is used to freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors. Initialization module 330 is used to initialize the network parameters of the differentiable PLDA network based on the network parameters of the generative PLDA network, and to unfreeze the network parameters of the voiceprint feature extraction network. Module 340 is used to calculate a dynamic smoothing weight for each training batch and construct a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and the second loss function of the differentiable PLDA network. The joint training module 350 is used to perform joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model.

[0067] Furthermore, the construction module 340 is used to calculate a dynamically smoothed weight for each training batch: Determine the average cosine similarity of all similar sample pairs within the current training batch; Determine the high-score average cosine similarity of all negative sample pairs in the current training batch whose average cosine similarity is greater than the preset threshold for negative samples; wherein, if there are no negative sample pairs in the current training batch that exceed the preset threshold for negative samples, then set the high-score average cosine similarity to the preset threshold for negative samples. Based on the mean cosine similarity, the high-score mean cosine similarity, and the preset performance target threshold, the optimization tendency score is determined. The optimization tendency score is processed based on a nonlinear mapping function to obtain instantaneous weights. Then, the instantaneous weights are processed by an exponential moving average to obtain the dynamic smooth weights of the current training batch.

[0068] Furthermore, the construction module 340 is used to determine the optimization tendency score using the following formula: ; in, To optimize the propensity score, For high-scoring mean cosine similarity, The average cosine similarity, Set a threshold for negative samples. A preset threshold is set for positive samples. as well as These are positive weight hyperparameters.

[0069] Furthermore, the construction module 340 determines the second loss function through the following steps: The embedding vector pairs from the training batch are input into the differentiable PLDA network to determine the log-likelihood ratio score of the homology probability. The log-likelihood ratio score is mapped using the Sigmoid function to determine the probability value; Based on the probability value, the true label of the sample pair, and the binary cross-entropy formula, the second loss function is determined.

[0070] Furthermore, the construction module 340 determines the hybrid loss function using the following formula:

[0071] in, For a mixed loss function, For the first loss function, For the second loss function, For dynamic smoothing weights, For 1- .

[0072] Furthermore, module 340 constructs training batches in the following manner: Training batches are constructed based on PK sampling strategy, active data augmentation strategy and active data generation strategy to ensure that the batch contains a sufficient number of low-scoring positive samples and / or high-scoring negative samples.

[0073] Furthermore, the joint training module 350 is used for speaker recognition based on the speaker recognition model: Preprocessing and acoustic feature extraction are performed on the speech data to be recognized; The acoustic features are input into the speaker recognition model's voiceprint feature extraction network, and a query embedding vector is output. The query embedding vector is paired with one or more reference embedding vectors, and each pair of embedding vectors is input into the differentiable PLDA network of the speaker recognition model to determine a similarity score. Based on the similarity score, the system outputs whether the speech data to be recognized and the target speaker are the same person.

[0074] This application provides a speaker recognition model training device based on dynamic weighted hybrid loss. The speaker recognition model training device includes: a feature pre-training module, used to train a voiceprint feature extraction network using a first loss function; wherein the first loss function is a loss function based on angular margin; a generative PLDA network training module, used to freeze the network parameters of the voiceprint feature extraction network, extract embedding vectors of training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors; an initialization module, used to initialize the network parameters of a differentiable PLDA network based on the network parameters of the generative PLDA network, and unfreeze the network parameters of the voiceprint feature extraction network; a construction module, used to calculate a dynamic smoothing weight for each training batch, and construct a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and a second loss function of the differentiable PLDA network; and a joint training module, used to perform joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model. By introducing a "hybrid loss function" in the joint optimization stage, which simultaneously retains the first loss function based on angle margin and the second loss function based on differentiable PLDA, an information feedback loop from the back end to the front end is constructed, enabling the final model to achieve better overall performance on multiple key indicators, thereby improving the accuracy of speaker recognition.

[0075] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0076] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 as well as Figure 2 The steps of the speaker recognition model training method based on dynamic weighted hybrid loss in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0077] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2The speaker recognition model training method steps based on dynamic weighted hybrid loss in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0078] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0082] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a speaker recognition model based on dynamic weighted hybrid loss, characterized in that, The speaker recognition model training method includes: The voiceprint feature extraction network is trained using a first loss function; wherein, the first loss function is a loss function based on angular margin. Freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of the training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors. The network parameters of the differentiable PLDA network are initialized based on the network parameters of the generative PLDA network, and the network parameters of the voiceprint feature extraction network are unfrozen. For each training batch, a dynamic smoothing weight is calculated, and a hybrid loss function consisting of the first loss function and the second loss function of the differentiable PLDA network is constructed based on the dynamic smoothing weight. The speaker recognition model is obtained by jointly backpropagating the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function, so as to enable speaker recognition based on the speaker recognition model.

2. The speaker recognition model training method according to claim 1, characterized in that, The calculation of a dynamic smoothing weight for each training batch includes: Determine the average cosine similarity of all similar sample pairs within the current training batch; Determine the high-score average cosine similarity of all negative sample pairs in the current training batch whose average cosine similarity is greater than the preset threshold for negative samples; wherein, if there are no negative sample pairs in the current training batch that exceed the preset threshold for negative samples, then set the high-score average cosine similarity to the preset threshold for negative samples. Based on the mean cosine similarity, the high-score mean cosine similarity, and the preset performance target threshold, the optimization tendency score is determined. The optimization tendency score is processed based on a nonlinear mapping function to obtain instantaneous weights. Then, the instantaneous weights are processed by an exponential moving average to obtain the dynamic smooth weights of the current training batch.

3. The speaker recognition model training method according to claim 2, characterized in that, The optimization tendency score is determined using the following formula: ； in, To optimize the propensity score, For high-scoring mean cosine similarity, The average cosine similarity, Set a threshold for negative samples. A preset threshold is set for positive samples. as well as These are positive weight hyperparameters.

4. The speaker recognition model training method according to claim 1, characterized in that, The second loss function is determined through the following steps: The embedding vector pairs from the training batch are input into the differentiable PLDA network to determine the log-likelihood ratio score of the homology probability. The log-likelihood ratio score is mapped using the Sigmoid function to determine the probability value; Based on the probability value, the true label of the sample pair, and the binary cross-entropy formula, the second loss function is determined.

5. The speaker recognition model training method according to claim 1, characterized in that, The hybrid loss function is determined by the following formula: in, For a mixed loss function, For the first loss function, For the second loss function, For dynamic smoothing weights, For 1- .

6. The method according to claim 1, characterized in that, The training batches are constructed using the following method: Training batches are constructed based on PK sampling strategy, active data augmentation strategy and active data generation strategy to ensure that the batch contains a sufficient number of low-scoring positive samples and / or high-scoring negative samples.

7. The speaker recognition model training method according to claim 1, characterized in that, The speaker recognition based on the speaker recognition model includes: Preprocessing and acoustic feature extraction are performed on the speech data to be recognized; The acoustic features are input into the speaker recognition model's voiceprint feature extraction network, and a query embedding vector is output. The query embedding vector is paired with one or more reference embedding vectors, and each pair of embedding vectors is input into the differentiable PLDA network of the speaker recognition model to determine a similarity score. Based on the similarity score, the system outputs whether the speech data to be recognized and the target speaker are the same person.

8. A speaker recognition model training device based on dynamic weighted hybrid loss, characterized in that, The speaker recognition model training device includes: The feature pre-training module is used to train the voiceprint feature extraction network using a first loss function; wherein, the first loss function is a loss function based on angular margin. The generative PLDA network training module is used to freeze the network parameters of the voiceprint feature extraction network, extract the embedding vectors of the training set samples based on the voiceprint feature extraction network, and train the generative PLDA network based on the embedding vectors. An initialization module is used to initialize the network parameters of the differentiable PLDA network based on the network parameters of the generative PLDA network, and to unfreeze the network parameters of the voiceprint feature extraction network. The construction module is used to calculate a dynamic smoothing weight for each training batch and construct a hybrid loss function based on the dynamic smoothing weight, which includes the first loss function and the second loss function of the differentiable PLDA network. The joint training module is used to perform joint backpropagation updates on the voiceprint feature extraction network and the differentiable PLDA network based on the hybrid loss function to obtain the speaker recognition model, so as to enable speaker recognition based on the speaker recognition model.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the speaker recognition model training method based on any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the speaker recognition model training method based on dynamic weighted hybrid loss as described in any one of claims 1 to 7.

Images