Voiceprint recognition model training method, device, equipment and medium

Abstract

The embodiments of the present invention provide a method, apparatus, device, and medium for training a voiceprint recognition model, belonging to the field of voiceprint recognition technology. The method includes using a feature extraction layer of a model to extract features from initial audio data to obtain initial sound features; using a voiceprint embedding layer of the model to extract characterizations of the initial sound features to obtain initial sound representation information; using an accent recognition layer of the model to perform accent recognition based on the initial sound representation information to obtain a first classification result; using an accent loss layer of the model to perform loss calculation based on the first classification result and an accent label to obtain a first loss value; using an identity recognition layer of the model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; using a voiceprint loss layer of the model to perform loss calculation based on the second classification result and the speaker label to obtain a second loss value; and training a voiceprint recognition model based on the first loss value and the second loss value to obtain a target recognition model.

Description

Technical Field

[0001] The present invention relates to the field of voiceprint recognition technology, and in particular to a training method, device, equipment and medium for a voiceprint recognition model. Background Art

[0002] In the insurance and financial sectors, voiceprint recognition technology is a crucial tool for accurately verifying and confirming the speaker's identity when users consult or conduct transactions over the phone. This technology relies on precise analysis of the user's voice characteristics to ensure that only authorized personnel can access sensitive information or perform critical operations. However, while highly effective, voiceprint recognition technology exhibits significant limitations when dealing with users with accents. Existing algorithms are inadequate at distinguishing between different accents, making it difficult for the system to effectively identify users with different accents despite having similar voices. For example, even if two people have very similar voices, they may be mistaken for the same person because their different accents cannot be recognized. This situation can lead to misjudgment, misinformation, or security vulnerabilities. Summary of the Invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for training a voiceprint recognition model, aiming to solve the problem that related technologies have difficulty distinguishing users with similar timbres but different accents when facing users with different accents.

[0004] In a first aspect, an embodiment of the present invention provides a method for training a voiceprint recognition model, comprising:

[0005] Obtaining training data, the training data comprising initial audio data and accent labels and speaker labels corresponding to the initial audio data;

[0006] Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features;

[0007] Using the voiceprint embedding layer of the voiceprint recognition model to perform voiceprint characterization extraction on the initial sound features to obtain initial sound characterization information;

[0008] Using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result;

[0009] Using the accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value;

[0010] Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result;

[0011] Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0012] The voiceprint recognition model is trained according to the first loss value and the second loss value to obtain a target recognition model.

[0013] In a second aspect, an embodiment of the present invention provides a training device for a voiceprint recognition model, comprising:

[0014] A data acquisition module, configured to obtain training data, wherein the training data includes initial audio data and accent labels and speaker labels corresponding to the initial audio data;

[0015] A feature extraction module, configured to extract sound features from the initial audio data using the feature extraction layer of the voiceprint recognition model to obtain initial sound features;

[0016] A voiceprint embedding module, configured to extract voiceprint representations of the initial voice features using the voiceprint embedding layer of the voiceprint recognition model to obtain initial voice representation information;

[0017] an accent recognition module, configured to utilize the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result;

[0018] an accent loss module, configured to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value;

[0019] An identity recognition module, configured to utilize the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result;

[0020] a voiceprint loss module, configured to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0021] A model training module is used to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0022] In a third aspect, an embodiment of the present invention further provides a terminal device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the training method of any voiceprint recognition model provided in the specification of the present invention are implemented.

[0023] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any voiceprint recognition model training method provided in the specification of the present invention.

[0024] An embodiment of the present invention provides a training method, apparatus, device and medium for a voiceprint recognition model, the method comprising: obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data; using a feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; using a voiceprint embedding layer of the voiceprint recognition model to extract voiceprint representations from the initial sound features to obtain initial sound representation information; using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; using the accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value; using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation based on the second classification result and the speaker label to obtain a second loss value; and training the voiceprint recognition model based on the first loss value and the second loss value to obtain a target recognition model. This method incorporates a training loss to distinguish different accents during the training process of the voiceprint recognition model. This strengthens the model's ability to learn accent differences during training. After training, the voiceprint recognition model can not only identify the speaker's voiceprint, but also improve its ability to recognize accent differences. Consequently, when two people with similar voices but different accents are used for recognition, the voiceprint recognition model can simultaneously and comprehensively judge the voice characteristics and accent characteristics to distinguish them as two different people, thereby improving recognition accuracy and addressing the inability of existing algorithms to effectively utilize accent differences. This method adds a new judgment dimension to distinguishing the identities of similar speakers. It also solves the problem that related technologies have difficulty distinguishing users with similar voices but different accents when faced with users with different accents. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 A flowchart of a method for training a voiceprint recognition model provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the model structure of a voiceprint recognition model provided by an embodiment of the present invention;

[0028] Figure 3 for Figure 1 A flowchart of sub-step S108 of the method for training the voiceprint recognition model;

[0029] Figure 4 A schematic diagram of the module structure of a voiceprint recognition model training device provided by an embodiment of the present invention;

[0030] Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0033] It should be understood that the terms used in this specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0034] Embodiments of the present invention provide a method, apparatus, device, and medium for training a voiceprint recognition model. The voiceprint recognition model training method can be applied to a terminal device, such as a tablet computer, laptop computer, desktop computer, personal digital assistant, wearable device, or other electronic device. The terminal device can also be a server or a server cluster.

[0035] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a method for training a voiceprint recognition model provided by an embodiment of the present invention.

[0037] like Figure 1 As shown, the training method of the voiceprint recognition model includes steps S101 to S108.

[0038] Step S101: Obtain training data, where the training data includes initial audio data and accent labels and speaker labels corresponding to the initial audio data.

[0039] Exemplarily, suitable audio data sources are selected, which may be public audio databases, social media platforms, podcasts, telephone recordings, etc. Ensure that the audio quality of these sources is good and covers a variety of accents and speakers. Each audio segment is analyzed to determine the speaker's accent type (such as Mandarin, Cantonese, English, French, etc.), and a corresponding accent label is assigned to each initial audio data segment, and a unique identifier is created for the speaker in each initial audio data segment, which can be classified according to the speaker's identity card or speaker ID. Finally, the initial audio data, accent label, and speaker label are sorted to ensure that the label information corresponding to each audio file is accurate, thereby obtaining training data containing the initial audio data and its corresponding accent label and speaker label.

[0040] Step S102: Using the feature extraction layer of the voiceprint recognition model, extract sound features from the initial audio data to obtain initial sound features.

[0041] For example, Figure 2 As shown in the figure, the voiceprint recognition model includes feature extraction layer, voiceprint embedding layer, accent recognition layer, accent loss layer, identity recognition layer, and voiceprint loss layer.

[0042] Exemplarily, the feature extraction layer of the voiceprint recognition model, such as Mel Filter-Banks, Mel-frequency cepstral coefficients (MFCC), or self-supervised features WavLM, data2vec, etc., is used to extract sound features from the initial audio data, thereby obtaining initial sound features corresponding to the initial audio data.

[0043] Step S103: Using the voiceprint embedding layer of the voiceprint recognition model, extract the voiceprint representation of the initial sound features to obtain initial sound representation information.

[0044] For example, at the voiceprint embedding layer, the voiceprint recognition model maps the input initial sound features to a vector representation in a high-dimensional space, thereby obtaining initial sound representation information, which captures the unique acoustic characteristics of the speaker in the initial audio data.

[0045] Step S104: Using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result.

[0046] Exemplarily, the initial sound representation information is input into the accent recognition layer of the voiceprint recognition model, and then the accent recognition layer processes the initial sound representation information through a series of neural network layers (such as fully connected layers, activation functions, multi-layer perceptrons, etc.), and analyzes the input initial sound representation information according to the parameters learned during the training process, thereby outputting a first classification result, which includes a predicted accent label and a predicted probability corresponding to the predicted accent label.

[0047] Step S105: Using the accent loss layer of the voiceprint recognition model, a loss calculation is performed according to the first classification result and the accent label to obtain a first loss value.

[0048] Exemplarily, a loss function, such as a cross-entropy loss function, is determined, and then the accent loss layer uses the cross-entropy loss function to perform loss calculation based on the first classification result and the accent label, thereby obtaining a first loss value.

[0049] Step S106: Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result.

[0050] Exemplarily, the initial sound representation information is input into the identity recognition layer of the voiceprint recognition model, and then the identity recognition layer processes the initial sound representation information through a series of neural network layers (such as fully connected layers, activation functions, multi-layer perceptrons, etc.), and analyzes the input initial sound representation information according to the parameters learned during the training process, thereby outputting a second classification result, which includes the predicted speaker label and the predicted probability corresponding to the predicted speaker label.

[0051] Step S107: Utilize the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value.

[0052] Exemplarily, a loss function, such as a cross-entropy loss function, is determined, and then the voiceprint loss layer uses the cross-entropy loss function to perform loss calculation based on the second classification result and the speaker label, thereby obtaining a second loss value.

[0053] In some embodiments, the loss functions of the voiceprint loss layer and the accent loss layer are both Additive Angular Margin Softmax.

[0054] Exemplarily, the loss function is determined to be AAM-Softmax (Additive Angular Margin Softmax). AAM-Softmax is improved based on the traditional Softmax function, adding an additional angular margin to increase the boundary between categories. Then, the accent loss layer uses AAM-Softmax to calculate the loss based on the first classification result and the accent label, thereby obtaining the first loss value.

[0055] Exemplarily, the loss function is determined to be AAM-Softmax, and then the voiceprint loss layer uses AAM-Softmax to calculate the loss according to the second classification result and the speaker label, thereby obtaining a second loss value.

[0056] Specifically, AAM-Softmax introduces an angular margin to increase the distance between the correct category score and the scores of other categories. This design forces the model to more strictly distinguish different categories in the feature space. This is especially true when dealing with highly similar categories, thus providing support for subsequent significant improvements in the model's distinguishing ability.

[0057] Step S108: Perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0058] Exemplarily, a first weight corresponding to the first loss value and a second weight corresponding to the second loss value are set according to expert experience or historical experience, and then the first loss value and the second loss value are fused according to the first weight and the second weight to obtain a target loss value, and then the target loss value is compared with the preset loss value. When the target loss value is less than the preset loss value, the corresponding target recognition model is directly obtained; when the target loss value is greater than or equal to the preset loss value, the voiceprint recognition model is continuously trained using the training data until the target loss value is less than the preset loss value, or when the number of training rounds is equal to the preset rounds, the corresponding target recognition model is obtained.

[0059] In one embodiment, the voiceprint recognition model is trained according to the first loss value and the second loss value to obtain the target recognition model. Specifically, referring to Figure 3 , step S108 includes: sub-step S1081 to sub-step S1082.

[0060] Sub-step S1081, determining hyperparameters, and fusing the first loss value and the second loss value according to the hyperparameters to determine a target loss value corresponding to the voiceprint recognition model.

[0061] For example, a hyperparameter λ is introduced based on expert experience or historical experience to form the target loss value of training The voiceprint recognition model will pass this total loss function Optimize.

[0062]

[0063] in, represents the target loss value, λ represents the hyperparameter, λ can be set to 0.1, L spk Represents the second loss value, L acc Represents the first loss value.

[0064] Sub-step S1082: performing model training on the voiceprint recognition model according to the target loss value to obtain the target recognition model.

[0065] Exemplarily, the target loss value is compared with the preset loss value. When the target loss value is less than the preset loss value, the corresponding target recognition model is directly obtained; when the target loss value is greater than or equal to the preset loss value, the voiceprint recognition model is continuously trained using the training data until the target loss value is less than the preset loss value, or when the number of training rounds is equal to the preset number of rounds, the corresponding target recognition model is obtained.

[0066] Specifically, by precisely adjusting the hyperparameters, the training process of the voiceprint recognition model can be better balanced, so that the voiceprint recognition model can learn the user's accent characteristics while learning the voiceprint features, providing support for subsequent improvement of the model's user recognition accuracy.

[0067] In some embodiments, after obtaining the target recognition model, the method further includes: obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure; obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure; and determining a voiceprint accent enhancement representation model based on the first network structure, the first model parameters, the second network structure, and the second model parameters.

[0068] For example, the voiceprint recognition model can only recognize speaker labels that already exist in the training data, but the voiceprint embedding layer in the voiceprint recognition model can accurately obtain the speaker's accent characteristics while obtaining the speaker's speech characteristics. Therefore, this application extracts some network layers of the voiceprint recognition model to obtain a voiceprint accent enhanced representation model, thereby providing support for the reliability of subsequent voiceprint representations of different speakers.

[0069] Exemplarily, a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure are obtained, as well as a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure are obtained, and then a voiceprint representation model based on accent enhancement is composed according to the first network structure, the first model parameters, the second network structure and the second model parameters, that is, a voiceprint accent enhancement representation model is obtained, so that according to the voiceprint accent enhancement representation model, the user's initial audio data can be represented for speech features while the accent features can also be extracted, thereby ensuring the reliability of the voiceprint representation and providing good support for the subsequent accurate identification of the speaker.

[0070] For example, by training an accent-enhanced voiceprint recognition model, the characteristic discrimination of accents can be enhanced, thereby reducing the number of voiceprint recognition errors caused by failure to distinguish accents. In turn, the effective use of accent information can further alleviate the problem of the voiceprint recognition model incorrectly distinguishing users with similar timbres but different accents.

[0071] For example, after the voiceprint recognition model is trained, a voiceprint and accent enhancement model derived from the model can not only identify the speaker's voiceprint, but also improve the ability to identify accent differences. For example, when two people with similar voices but different accents are used for recognition, the voiceprint and accent enhancement model can comprehensively identify both the voice characteristics and the accent characteristics, thereby distinguishing them as two different people.

[0072] In some embodiments, after obtaining the voiceprint accent enhancement characterization model, the method further includes: obtaining target audio data corresponding to the target user; performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target sound characterization information corresponding to the target user; calculating distance information between the target sound characterization information and preset sound characterization information, and determining the user identity corresponding to the target user based on the distance information.

[0073] For example, the target user's target audio data is obtained through an online voice platform, and then the voiceprint and accent enhancement model is used to extract features from the target audio data to generate a target voice representation vector corresponding to the target user. The target voice representation vector can accurately reflect the target user's voiceprint characteristics and accent characteristics.

[0074] For example, preset sound representation information of known users is pre-stored, and then the distance information between the target sound representation information and the preset sound representation information is calculated using an appropriate distance measurement method (such as Euclidean distance, cosine similarity, etc.). The smaller the distance information, the higher the similarity.

[0075] For example, a suitable distance threshold is set to determine whether the target user matches a known user. The minimum value of the distance information is obtained and compared with the distance threshold. When the minimum value of the distance information is less than the distance threshold, the target user is considered to match the known user corresponding to the preset sound representation information, and the user identity corresponding to the target user is determined based on the user information corresponding to the known user. When the minimum value of the distance information is greater than or equal to the distance threshold, it is considered that no user matching the target user is found among the known users, and the user identity corresponding to the target user cannot be determined.

[0076] Specifically, by generating a target sound representation vector corresponding to the target audio data based on the voiceprint and accent enhanced representation model, it is possible to obtain the target user's voiceprint features while also taking into account the accent features, thereby solving the problem of incorrect recognition of speakers with similar timbre but different accents, thereby improving the accuracy of identity recognition of users with accents.

[0077] In some embodiments, after determining the user identity corresponding to the target user based on the distance information, the method further includes: obtaining the target identity corresponding to the target user, and determining the accuracy of the voiceprint and accent enhancement characterization model based on the target identity and the user identity; updating the target recognition model based on the accuracy to obtain the updated target recognition model; and updating the voiceprint and accent enhancement characterization model based on the updated target recognition model to obtain the updated voiceprint and accent enhancement characterization model.

[0078] Exemplarily, the target audio data is manually judged to obtain the target identity corresponding to the target user, and then the accuracy of the voice feature characterization by the voiceprint accent enhancement characterization model is determined based on whether the target identity and the user identity are the same.

[0079] Exemplarily, when the accuracy is lower than a preset value, the training data is retrieved, and the target recognition model is updated based on the retrieved training data to obtain an updated target recognition model.

[0080] Exemplarily, after obtaining the updated target recognition model, the first model parameters corresponding to the first network structure in the voiceprint and accent enhancement representation model and the second model parameters corresponding to the second network structure are obtained from the updated target recognition model, and then the parameters of the voiceprint and accent enhancement representation model are updated according to the first model parameters and the second model parameters, so as to obtain the updated voiceprint and accent enhancement representation model.

[0081] Specifically, by comparing the target identity and user identity of the target user, the representation effect of the current voiceprint and accent enhancement representation model can be accurately evaluated, and the voiceprint and accent enhancement representation model can be continuously updated through accuracy, so that it can better adapt to the needs of specific user groups, thereby improving the representation ability of the voiceprint and accent enhancement representation model, and providing support for subsequent user identity recognition.

[0082] In some embodiments, the target recognition model is updated according to the accuracy rate to obtain the updated target recognition model, including: when the accuracy rate is lower than a preset value, determining incremental data based on the data when the target identity and the user identity are inconsistent; incrementally training the target recognition model according to the incremental data, and then updating the target recognition model to obtain the updated target recognition model.

[0083] Exemplarily, a preset value is set according to actual needs, and when the accuracy rate is lower than the preset value, the target audio data corresponding to the target identity and the user identity are collected, and the target audio data is annotated to obtain the speaker label and accent label corresponding to the target audio data, thereby determining the incremental data based on the target audio data, the speaker label corresponding to the target audio data, and the accent label corresponding to the target audio data.

[0084] Exemplarily, after obtaining the incremental data, the target recognition model is incrementally trained based on the incremental data, thereby obtaining the incrementally trained target recognition model, that is, obtaining the updated target recognition model.

[0085] Specifically, incremental training can significantly reduce training time and enable the target recognition model to adapt to new data in a timely manner without retraining the entire target recognition model, thereby improving the efficiency of updating the target recognition model.

[0086] In some embodiments, after obtaining the voiceprint and accent enhanced representation model, the method further includes: determining a voice sample corresponding to the training of the initial speech recognition model, the voice sample including voice data and text information corresponding to the voice data; performing voiceprint representation processing on the voice data according to the voiceprint and accent enhanced representation model to obtain voice representation information corresponding to the voice data; performing acoustic feature mapping on the voice representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the voice data; performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information; performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data; performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value; and performing model training on the initial speech recognition model according to the language loss value to obtain a target speech recognition model.

[0087] For example, after obtaining a voiceprint accent enhancement representation model with accent enhancement, the voiceprint accent enhancement representation model can be used to provide support for subsequent improvement of a speech recognition model with speech, and then the voiceprint accent enhancement representation model can be used to capture speaker characteristics in speech data.

[0088] For example, based on the acoustic network of the initial speech recognition model, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a Transformer, the speech representation information obtained by the voiceprint and accent enhancement representation model is input into the acoustic network, which processes the speech representation information and extracts the target acoustic features.

[0089] For example, based on the language network of the initial speech recognition model, such as N-gram, or a more complex neural network language network (such as RNN, LSTM, Transformer, etc.), the text information is input into the language network, which uses the language network to learn the language rules in the text data and extract the target text features.

[0090] Exemplarily, the decoding layer of the initial speech recognition model may be a decoder based on the Viterbi algorithm or Beam Search, thereby utilizing the decoding layer to combine target acoustic features and target text features to perform language prediction and generate predicted text corresponding to the speech data.

[0091] For example, a loss function, such as cross-entropy loss or CTC (Connectionist Temporal Classification) loss, is determined for the language loss layer of the initial speech recognition model. The predicted text is compared with the text information to calculate a language loss value. The language loss value is then used to evaluate the prediction accuracy of the initial speech recognition model and provide error feedback during training. If the language loss value does not meet the requirements, the calculated language loss value is used to update the model parameters through backpropagation, and the initial speech recognition model is iteratively trained using speech samples until the target speech recognition model is obtained.

[0092] Specifically, the target speech recognition model is based on the voiceprint and accent enhanced representation model, thus providing strong support for the subsequent improvement of the speech recognition accuracy of users with accents.

[0093] An embodiment of the present invention provides a training method, apparatus, device and medium for a voiceprint recognition model, the method comprising: obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data; using a feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; using a voiceprint embedding layer of the voiceprint recognition model to extract voiceprint representations from the initial sound features to obtain initial sound representation information; using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; using the accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value; using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation based on the second classification result and the speaker label to obtain a second loss value; and training the voiceprint recognition model based on the first loss value and the second loss value to obtain a target recognition model. This method incorporates a training loss to distinguish different accents during the training process of the voiceprint recognition model. This strengthens the model's ability to learn accent differences during training. After training, the voiceprint recognition model can not only identify the speaker's voiceprint, but also improve its ability to recognize accent differences. Consequently, when two people with similar voices but different accents are used for recognition, the voiceprint recognition model can simultaneously and comprehensively judge the voice characteristics and accent characteristics to distinguish them as two different people, thereby improving recognition accuracy and addressing the inability of existing algorithms to effectively utilize accent differences. This method adds a new judgment dimension to distinguishing the identities of similar speakers. It also solves the problem that related technologies have difficulty distinguishing users with similar voices but different accents when faced with users with different accents.

[0094] See also Figure 4 , Figure 4A training device 200 for a voiceprint recognition model provided in an embodiment of the present application includes a data acquisition module 201, a feature extraction module 202, a voiceprint embedding module 203, an accent recognition module 204, an accent loss module 205, an identity recognition module 206, a voiceprint loss module 207, and a model training module 208, wherein the data acquisition module 201 is used to obtain training data, and the training data includes initial audio data and an accent label and a speaker label corresponding to the initial audio data; the feature extraction module 202 is used to use the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; the voiceprint embedding module 203 is used to use the voiceprint embedding layer of the voiceprint recognition model to extract voiceprint representations from the initial sound features to obtain initial sound features. characterization information; an accent recognition module 204, used to use the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound characterization information to obtain a first classification result; an accent loss module 205, used to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; an identity recognition module 206, used to use the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound characterization information to obtain a second classification result; a voiceprint loss module 207, used to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; a model training module 208, used to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0095] In some embodiments, during the process of training the voiceprint recognition model according to the first loss value and the second loss value to obtain the target recognition model, the model training module 208 performs:

[0096] Determining a hyperparameter, and fusing the first loss value and the second loss value according to the hyperparameter to determine a target loss value corresponding to the voiceprint recognition model;

[0097] The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

[0098] In some embodiments, after obtaining the target recognition model, the voiceprint recognition model training device 200 further performs:

[0099] Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure;

[0100] Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure;

[0101] A voiceprint accent enhancement representation model is determined according to the first network structure, the first model parameters, the second network structure, and the second model parameters.

[0102] In some embodiments, after obtaining the voiceprint accent enhancement representation model, the voiceprint recognition model training device 200 further performs:

[0103] Obtain target audio data corresponding to the target user;

[0104] Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target voice characterization information corresponding to the target user;

[0105] The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

[0106] In some embodiments, after determining the user identity corresponding to the target user according to the distance information, the voiceprint recognition model training device 200 further performs:

[0107] Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint and accent enhancement representation model based on the target identity and the user identity;

[0108] Performing a model update on the target recognition model according to the accuracy rate to obtain the updated target recognition model;

[0109] The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

[0110] In some embodiments, the voiceprint recognition model training device 200 performs the following steps during the process of updating the target recognition model according to the accuracy rate to obtain the updated target recognition model:

[0111] When the accuracy rate is lower than a preset value, incremental data is determined based on data when the target identity and the user identity are inconsistent;

[0112] Incremental training is performed on the target recognition model according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

[0113] In some embodiments, after obtaining the voiceprint accent enhancement representation model, the voiceprint recognition model training device 200 further performs:

[0114] Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data;

[0115] performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data;

[0116] Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data;

[0117] Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information;

[0118] Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data;

[0119] Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value;

[0120] The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

[0121] In some embodiments, the voiceprint recognition model training device 200 can be applied to a terminal device.

[0122] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the voiceprint recognition model training device 200 described above can refer to the corresponding process in the aforementioned voiceprint recognition model training method embodiment, and will not be repeated here.

[0123] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention.

[0124] like Figure 5 As shown, the terminal device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected via a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0125] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0126] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0127] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the embodiment of the present invention, and does not constitute a limitation on the terminal device to which the embodiment of the present invention is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0128] The processor is used to run the computer program stored in the memory, and implement any one of the voiceprint recognition model training methods provided by the embodiments of the present invention when executing the computer program.

[0129] In one embodiment, the processor is configured to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0130] Obtaining training data, the training data comprising initial audio data and accent labels and speaker labels corresponding to the initial audio data;

[0131] Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features;

[0132] Using the voiceprint embedding layer of the voiceprint recognition model to perform voiceprint characterization extraction on the initial sound features to obtain initial sound characterization information;

[0133] Using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result;

[0134] Using the accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value;

[0135] Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result;

[0136] Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0137] The voiceprint recognition model is trained according to the first loss value and the second loss value to obtain a target recognition model.

[0138] In some embodiments, during the process of training the voiceprint recognition model according to the first loss value and the second loss value to obtain the target recognition model, the processor 301 executes:

[0139] Determining a hyperparameter, and fusing the first loss value and the second loss value according to the hyperparameter to determine a target loss value corresponding to the voiceprint recognition model;

[0140] The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

[0141] In some implementations, after obtaining the target recognition model, the processor 301 further executes:

[0142] Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure;

[0143] Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure;

[0144] A voiceprint accent enhancement representation model is determined according to the first network structure, the first model parameters, the second network structure, and the second model parameters.

[0145] In some implementations, after obtaining the voiceprint accent enhancement representation model, the processor 301 further executes:

[0146] Obtain target audio data corresponding to the target user;

[0147] Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target voice characterization information corresponding to the target user;

[0148] The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

[0149] In some implementations, after determining the user identity corresponding to the target user according to the distance information, the processor 301 further performs:

[0150] Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint and accent enhancement representation model based on the target identity and the user identity;

[0151] Performing a model update on the target recognition model according to the accuracy rate to obtain the updated target recognition model;

[0152] The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

[0153] In some implementations, during the process of updating the target recognition model according to the accuracy rate to obtain the updated target recognition model, the processor 301 executes:

[0154] When the accuracy rate is lower than a preset value, incremental data is determined based on data when the target identity and the user identity are inconsistent;

[0155] Incremental training is performed on the target recognition model according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

[0156] In some implementations, after obtaining the voiceprint accent enhancement representation model, the processor 301 further executes:

[0157] Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data;

[0158] performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data;

[0159] Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data;

[0160] Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information;

[0161] Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data;

[0162] Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value;

[0163] The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

[0164] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the embodiment of the training method of the aforementioned voiceprint recognition model, and will not be repeated here.

[0165] An embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any voiceprint recognition model training method provided in the description of the embodiment of the present invention.

[0166] The storage medium may be an internal storage unit of the terminal device in the aforementioned embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc.

[0167] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0168] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0169] The serial numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope of protection of the claims.

Claims

1. A method for training a voiceprint recognition model, characterized in that: The method comprises: Obtaining training data, the training data comprising initial audio data and accent labels and speaker labels corresponding to the initial audio data; Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; Using the voiceprint embedding layer of the voiceprint recognition model to perform voiceprint characterization extraction on the initial sound features to obtain initial sound characterization information; Using the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; Using the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; Performing model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model; Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure; Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure; A voiceprint accent enhancement representation model is determined according to the first network structure, the first model parameters, the second network structure, and the second model parameters.

2. The method according to claim 1, characterized in that The performing model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model includes: Determining a hyperparameter, and fusing the first loss value and the second loss value according to the hyperparameter to determine a target loss value corresponding to the voiceprint recognition model; The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

3. The method according to claim 1, characterized in that After obtaining the voiceprint accent enhancement representation model, the method further includes: Obtain target audio data corresponding to the target user; Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target voice characterization information corresponding to the target user; The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

4. The method according to claim 3, characterized in that After determining the user identity corresponding to the target user according to the distance information, the method further includes: Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint and accent enhancement representation model based on the target identity and the user identity; Performing a model update on the target recognition model according to the accuracy rate to obtain the updated target recognition model; The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

5. The method according to claim 4, characterized in that The updating of the target recognition model according to the accuracy rate to obtain the updated target recognition model includes: When the accuracy rate is lower than a preset value, incremental data is determined based on data when the target identity and the user identity are inconsistent; Incremental training is performed on the target recognition model according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

6. The method according to claim 1, characterized in that After obtaining the voiceprint accent enhancement representation model, the method further includes: Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data; performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data; Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data; Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information; Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data; Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value; The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

7. A training device for a voiceprint recognition model, characterized in that: include: A data acquisition module, configured to obtain training data, wherein the training data includes initial audio data and accent labels and speaker labels corresponding to the initial audio data; A feature extraction module, configured to extract sound features from the initial audio data using the feature extraction layer of the voiceprint recognition model to obtain initial sound features; A voiceprint embedding module, configured to extract voiceprint representations of the initial voice features using the voiceprint embedding layer of the voiceprint recognition model to obtain initial voice representation information; an accent recognition module, configured to utilize the accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; an accent loss module, configured to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; An identity recognition module, configured to utilize the identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; a voiceprint loss module, configured to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; a model training module, configured to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model; After obtaining the target recognition model, the voiceprint recognition model training device is further used to: Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and first model parameters corresponding to the first network structure; Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and second model parameters corresponding to the second network structure; A voiceprint accent enhancement representation model is determined according to the first network structure, the first model parameters, the second network structure, and the second model parameters.

8. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the method for training the voiceprint recognition model according to any one of claims 1 to 6 when executing the computer program.

9. A computer storage medium for computer storage, characterized in that: The computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the voiceprint recognition model training method according to any one of claims 1 to 6.

Images