Voiceprint recognition model training method and device, equipment and medium

Abstract

The embodiment of the invention provides a voiceprint recognition model training method and device, equipment and a medium, and belongs to the technical field of voiceprint recognition. The method comprises the following steps: performing feature extraction on initial audio data by using a feature extraction layer of a model to obtain initial sound features; the voiceprint embedding layer of the model is utilized to carry out representation extraction on the initial sound features to obtain initial sound representation information; performing accent recognition according to the initial sound representation information by using an accent recognition layer of the model to obtain a first classification result; performing loss calculation according to the first classification result and the accent label by using an accent loss layer of the model to obtain a first loss value; carrying out speaker recognition according to the initial sound representation information by using an identity recognition layer of the model to obtain a second classification result; performing loss calculation according to the second classification result and the speaker label by using a voiceprint loss layer of the model to obtain a second loss value; and performing model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a training method, device, equipment and medium for a voiceprint recognition model. Background Art

[0002] In the insurance or financial sectors, voiceprint recognition technology is an important tool for accurately verifying and confirming the identity of the speaker when the user consults or trades over the phone. This technology relies on the precise analysis of the user's voice characteristics to ensure that only authorized personnel can access sensitive information or perform key operations. However, although voiceprint recognition technology is efficient, it shows obvious limitations when dealing with users with accents. Since existing algorithms are inadequate in distinguishing different accents, this may make it difficult for the system to effectively identify users who have different accents despite similar timbre. For example, even if two people have very similar timbre, they may be mistaken for the same person because their different accents cannot be recognized. This situation may lead to misjudgment, resulting in miscommunication of information or security vulnerabilities. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for training a voiceprint recognition model, aiming to solve the problem that related technologies are difficult to distinguish between users with similar timbres but different accents when facing users with different accents.

[0004] In a first aspect, an embodiment of the present invention provides a method for training a voiceprint recognition model, comprising:

[0005] Obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data;

[0006] Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features;

[0007] Using the voiceprint embedding layer of the voiceprint recognition model to extract the voiceprint representation of the initial sound feature to obtain initial sound representation information;

[0008] Using the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result;

[0009] Using the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value;

[0010] Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result;

[0011] Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0012] The voiceprint recognition model is trained according to the first loss value and the second loss value to obtain a target recognition model.

[0013] In a second aspect, an embodiment of the present invention provides a training device for a voiceprint recognition model, comprising:

[0014] A data acquisition module, used to obtain training data, wherein the training data includes initial audio data and an accent label and a speaker label corresponding to the initial audio data;

[0015] A feature extraction module, used to extract sound features from the initial audio data using the feature extraction layer of the voiceprint recognition model to obtain initial sound features;

[0016] A voiceprint embedding module, configured to extract voiceprint representation of the initial sound feature using the voiceprint embedding layer of the voiceprint recognition model to obtain initial sound representation information;

[0017] an accent recognition module, configured to use the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result;

[0018] an accent loss module, configured to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value;

[0019] An identity recognition module, configured to use the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result;

[0020] A voiceprint loss module, configured to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0021] A model training module is used to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0022] In a third aspect, an embodiment of the present invention further provides a terminal device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any method for training a voiceprint recognition model provided in the specification of the present invention are implemented.

[0023] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any voiceprint recognition model training method provided in the specification of the present invention.

[0024] The embodiment of the present invention provides a training method, device, equipment and medium for a voiceprint recognition model, the method comprising: obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data; using a feature extraction layer of a voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; using a voiceprint embedding layer of a voiceprint recognition model to extract voiceprint representations from the initial sound features to obtain initial sound representation information; using an accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; using an accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value; using an identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; using a voiceprint loss layer of the voiceprint recognition model to perform loss calculation based on the second classification result and the speaker label to obtain a second loss value; and training the voiceprint recognition model based on the first loss value and the second loss value to obtain a target recognition model. This method adds a training loss for distinguishing different accents during the training process of the voiceprint recognition model, which can enhance the model's ability to learn accent differences during the training process. After the training is completed, the voiceprint recognition model can improve its ability to recognize accent differences while judging the speaker's voiceprint. When the voices of two people with similar voices but different accents are used for recognition, the voiceprint recognition model can comprehensively judge the voice characteristics and accent characteristics at the same time, thereby distinguishing that they are two different people, thereby improving the recognition accuracy and improving the defect that the existing algorithm cannot effectively utilize accent differences. This method adds a new judgment dimension to distinguish the identities of similar speakers. It also solves the problem that related technologies are difficult to distinguish between users with similar voices but different accents when facing users with different accents. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of a method for training a voiceprint recognition model provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the model structure of a voiceprint recognition model provided by an embodiment of the present invention;

[0028] Figure 3 for Figure 1 A flowchart of sub-step S108 of the method for training a voiceprint recognition model;

[0029] Figure 4 A schematic diagram of the module structure of a training device for a voiceprint recognition model provided by an embodiment of the present invention;

[0030] Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0033] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0034] The embodiments of the present invention provide a method, device, equipment and medium for training a voiceprint recognition model. The method for training a voiceprint recognition model can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device. The terminal device can be a server or a server cluster.

[0035] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a method for training a voiceprint recognition model provided by an embodiment of the present invention.

[0037] like Figure 1 As shown, the training method of the voiceprint recognition model includes steps S101 to S108.

[0038] Step S101: obtaining training data, wherein the training data includes initial audio data and an accent label and a speaker label corresponding to the initial audio data.

[0039] Exemplarily, suitable audio data sources are selected, which may be public audio databases, social media platforms, podcasts, telephone recordings, etc. Ensure that the audio quality of these sources is good and covers a variety of accents and speakers. Each audio segment is analyzed to determine the speaker's accent type (such as Mandarin, Cantonese, English, French, etc.), and a corresponding accent label is assigned to each initial audio data segment, and a unique identifier is created for the speaker in each initial audio data segment, which can be classified according to the speaker's ID card or speaker ID. Finally, the initial audio data, accent labels, and speaker labels are sorted to ensure that the label information corresponding to each audio file is accurate, thereby obtaining training data containing the initial audio data and its corresponding accent labels and speaker labels.

[0040] Step S102: Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features.

[0041] For example, Figure 2 As shown, the voiceprint recognition model includes a feature extraction layer, a voiceprint embedding layer, an accent recognition layer, an accent loss layer, an identity recognition layer, and a voiceprint loss layer.

[0042] Exemplarily, the feature extraction layer of the voiceprint recognition model, such as Mel Filter-Banks, Mel Frequency Cepstral Coefficients (MFCC), or self-supervised features WavLM, data2vec, etc., is used to extract sound features from the initial audio data, thereby obtaining initial sound features corresponding to the initial audio data.

[0043] Step S103: using the voiceprint embedding layer of the voiceprint recognition model to extract voiceprint representation of the initial sound features to obtain initial sound representation information.

[0044] Exemplarily, at the voiceprint embedding layer, the voiceprint recognition model maps the input initial sound features to a vector representation in a high-dimensional space, thereby obtaining initial sound representation information, which captures the unique acoustic characteristics of the speaker in the initial audio data.

[0045] Step S104: using the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result.

[0046] Exemplarily, the initial sound representation information is input into the accent recognition layer of the voiceprint recognition model, and then the accent recognition layer processes the initial sound representation information through a series of neural network layers (e.g., fully connected layers, activation functions, multi-layer perceptrons, etc.), and parses the input initial sound representation information according to the parameters learned during the training process, thereby outputting a first classification result, which includes a predicted accent label and a predicted probability corresponding to the predicted accent label.

[0047] Step S105: Using the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value.

[0048] Exemplarily, a loss function, such as a cross-entropy loss function, is determined, and then the accent loss layer uses the cross-entropy loss function to perform loss calculation based on the first classification result and the accent label, thereby obtaining a first loss value.

[0049] Step S106: Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result.

[0050] Exemplarily, the initial sound characterization information is input into the identity recognition layer of the voiceprint recognition model, and then the initial sound characterization information is processed by a series of neural network layers (such as fully connected layers, activation functions, multi-layer perceptrons, etc.) in the identity recognition layer, and the input initial sound characterization information is parsed according to the parameters learned during the training process, so as to output a second classification result, which includes the predicted speaker label and the predicted probability corresponding to the predicted speaker label.

[0051] Step S107: Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value.

[0052] Exemplarily, a loss function, such as a cross-entropy loss function, is determined, and then the voiceprint loss layer uses the cross-entropy loss function to perform loss calculation according to the second classification result and the speaker label, thereby obtaining a second loss value.

[0053] In some implementations, the loss functions of the voiceprint loss layer and the accent loss layer are both Additive Angular Margin Softmax.

[0054] Exemplarily, the loss function is determined to be AAM-Softmax (Additive Angular Margin Softmax). AAM-Softmax is improved on the basis of the traditional Softmax function, and an additional angular margin is added to increase the boundary between categories. Then the accent loss layer uses AAM-Softmax to calculate the loss according to the first classification result and the accent label, so as to obtain the first loss value.

[0055] Exemplarily, the loss function is determined to be AAM-Softmax, and then the voiceprint loss layer uses AAM-Softmax to calculate the loss according to the second classification result and the speaker label, so as to obtain the second loss value.

[0056] Specifically, AAM-Softmax introduces an angle interval to increase the distance between the correct category score and the scores of other categories. This design forces the model to more strictly distinguish different categories in the feature space. This is especially true when dealing with highly similar categories, thus providing support for the subsequent significant improvement of the model's distinguishing ability.

[0057] Step S108: training the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0058] Exemplarily, a first weight corresponding to the first loss value and a second weight corresponding to the second loss value are set according to expert experience or historical experience, and then the first loss value and the second loss value are fused according to the first weight and the second weight to obtain a target loss value, and then the target loss value is compared with the preset loss value. When the target loss value is less than the preset loss value, the corresponding target recognition model is directly obtained; when the target loss value is greater than or equal to the preset loss value, the training data is continued to be used to train the voiceprint recognition model until the target loss value is less than the preset loss value, or when the number of training rounds is equal to the preset rounds, the corresponding target recognition model is obtained.

[0059] In one embodiment, the voiceprint recognition model is trained according to the first loss value and the second loss value to obtain the target recognition model. Specifically, referring to Figure 3 , step S108 includes: sub-step S1081 to sub-step S1082.

[0060] Sub-step S1081, determining a hyperparameter, and fusing the first loss value and the second loss value according to the hyperparameter to determine a target loss value corresponding to the voiceprint recognition model.

[0061] For example, a hyperparameter λ is introduced based on expert experience or historical experience to form the target loss value of training The voiceprint recognition model will pass this total loss function Make optimizations.

[0062]

[0063] in, represents the target loss value, λ represents the hyperparameter, λ can be set to 0.1, L spk Represents the second loss value, L acc Represents the first loss value.

[0064] Sub-step S1082: training the voiceprint recognition model according to the target loss value to obtain the target recognition model.

[0065] Exemplarily, the target loss value is compared with the preset loss value. When the target loss value is less than the preset loss value, the corresponding target recognition model is directly obtained; when the target loss value is greater than or equal to the preset loss value, the voiceprint recognition model is continuously trained using the training data until the target loss value is less than the preset loss value, or when the number of training rounds is equal to the preset number of rounds, the corresponding target recognition model is obtained.

[0066] Specifically, by precisely adjusting the hyperparameters, the training process of the voiceprint recognition model can be better balanced, so that the voiceprint recognition model can learn the user's accent characteristics while learning the voiceprint features, providing support for subsequently improving the model's user recognition accuracy.

[0067] In some embodiments, after obtaining the target recognition model, the method further includes: obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and a first model parameter corresponding to the first network structure; obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and a second model parameter corresponding to the second network structure; and determining a voiceprint accent enhancement characterization model based on the first network structure, the first model parameter, the second network structure and the second model parameter.

[0068] Exemplarily, the voiceprint recognition model can only recognize speaker labels that already exist in the training data, but the voiceprint embedding layer in the voiceprint recognition model can accurately obtain the speaker's accent characteristics while obtaining the speaker's speaking characteristics. Therefore, this application extracts some network layers of the voiceprint recognition model to obtain a voiceprint accent enhanced representation model, thereby providing support for the reliability of subsequent voiceprint representations of different speakers.

[0069] Exemplarily, a first network structure corresponding to a feature extraction layer in a target recognition model and a first model parameter corresponding to the first network structure are obtained, as well as a second network structure corresponding to a voiceprint embedding layer in the target recognition model and a second model parameter corresponding to the second network structure are obtained, and then a voiceprint characterization model based on accent enhancement is composed according to the first network structure, the first model parameter, the second network structure and the second model parameter, that is, a voiceprint accent enhancement characterization model is obtained, so that according to the voiceprint accent enhancement characterization model, the user's initial audio data can be characterized for speech features while the accent features can be extracted, thereby ensuring the reliability of the voiceprint characterization and providing good support for the subsequent accurate identification of the speaker.

[0070] For example, by training a voiceprint recognition model with accent enhancement, the characteristic differentiation of accents can be enhanced, thereby reducing the number of voiceprint recognition errors caused by failure to distinguish accents. Effective use of accent information can then further alleviate the problem of voiceprint recognition models incorrectly distinguishing users with similar timbres but different accents.

[0071] For example, after the training of the voiceprint recognition model is completed, the voiceprint accent enhancement representation model obtained based on the voiceprint recognition model can improve the recognition ability of accent differences while judging the voiceprint of the speaker. For example, when the voices of two people with similar speaking timbre but different accents are used for recognition, the voiceprint accent enhancement representation model can comprehensively judge the voice characteristics and accent characteristics at the same time, thereby distinguishing them as two different people.

[0072] In some embodiments, after obtaining the voiceprint accent enhancement characterization model, the method further includes: obtaining target audio data corresponding to the target user; performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target sound characterization information corresponding to the target user; calculating distance information between the target sound characterization information and preset sound characterization information, and determining the user identity corresponding to the target user based on the distance information.

[0073] For example, the target audio data of the target user is obtained through the online voice platform, and then the target audio data is feature extracted using the voiceprint and accent enhancement representation model to generate a target voice representation vector corresponding to the target user. The target voice representation vector can accurately reflect the voiceprint features and accent features of the target user.

[0074] Exemplarily, preset sound representation information of known users is pre-stored, so as to calculate the distance information between the target sound representation information and the preset sound representation information using a suitable distance measurement method (such as Euclidean distance, cosine similarity, etc.). The smaller the distance information, the higher the similarity.

[0075] Exemplarily, a suitable distance threshold is set to determine whether the target user matches a known user. The minimum value of the distance information is obtained, and the minimum value of the distance information is compared with the distance threshold. When the minimum value of the distance information is less than the distance threshold, it is considered that the target user matches the known user corresponding to the preset sound characterization information, and the user identity corresponding to the target user is determined based on the user information corresponding to the known user. When the minimum value of the distance information is greater than or equal to the distance threshold, it is considered that no user matching the target user is found among the known users, and thus the user identity corresponding to the target user cannot be determined.

[0076] Specifically, by generating a target sound representation vector corresponding to the target audio data based on the voiceprint and accent enhanced representation model, it is possible to obtain the voiceprint features of the target user while also taking into account the accent features, thereby solving the problem of incorrect recognition of speakers with similar timbres but different accents, thereby improving the accuracy of identity recognition of users with accents.

[0077] In some embodiments, after determining the user identity corresponding to the target user according to the distance information, the method further includes: obtaining the target identity corresponding to the target user, and determining the accuracy of the voiceprint and accent enhancement characterization model according to the target identity and the user identity; updating the target recognition model according to the accuracy to obtain the updated target recognition model; updating the voiceprint and accent enhancement characterization model according to the updated target recognition model to obtain the updated voiceprint and accent enhancement characterization model.

[0078] Exemplarily, the target audio data is manually judged to obtain the target identity corresponding to the target user, and then the accuracy of the voiceprint accent enhancement characterization model in characterizing speech features is determined based on whether the target identity and the user identity are the same.

[0079] Exemplarily, when the accuracy is lower than a preset value, the training data is retrieved, and the target recognition model is updated according to the retrieved training data, thereby obtaining an updated target recognition model.

[0080] Exemplarily, after obtaining the updated target recognition model, the first model parameters corresponding to the first network structure in the voiceprint and accent enhancement characterization model and the second model parameters corresponding to the second network structure are re-obtained from the updated target recognition model, and then the parameters of the voiceprint and accent enhancement characterization model are updated according to the first model parameters and the second model parameters, so as to obtain the updated voiceprint and accent enhancement characterization model.

[0081] Specifically, by comparing the target identity and user identity of the target user, the representation effect of the current voiceprint and accent enhancement representation model can be accurately evaluated, so that the voiceprint and accent enhancement representation model can be continuously updated through accuracy, so that it can better adapt to the needs of specific user groups, thereby improving the representation ability of the voiceprint and accent enhancement representation model, and then providing support for subsequent user identity recognition.

[0082] In some embodiments, the target recognition model is updated according to the accuracy rate to obtain the updated target recognition model, including: when the accuracy rate is lower than a preset value, determining incremental data based on data when the target identity and the user identity are inconsistent; incrementally training the target recognition model according to the incremental data, and then updating the target recognition model to obtain the updated target recognition model.

[0083] Exemplarily, a preset value is set according to actual needs, and when the accuracy rate is lower than the preset value, the target audio data corresponding to the target identity and the user identity are collected, and the target audio data is annotated to obtain the speaker label and accent label corresponding to the target audio data, thereby determining the incremental data based on the target audio data, the speaker label corresponding to the target audio data, and the accent label corresponding to the target audio data.

[0084] Exemplarily, after obtaining the incremental data, the target recognition model is incrementally trained according to the incremental data, thereby obtaining the incrementally trained target recognition model, that is, obtaining the updated target recognition model.

[0085] Specifically, incremental training can significantly reduce training time and enable the target recognition model to adapt to new data in a timely manner without retraining the entire target recognition model, thereby improving the efficiency of updating the target recognition model.

[0086] In some embodiments, after obtaining the voiceprint accent enhancement representation model, the method further includes: determining a voice sample corresponding to the initial speech recognition model training, the voice sample including voice data and text information corresponding to the voice data; performing voiceprint representation processing on the voice data according to the voiceprint accent enhancement representation model to obtain voice representation information corresponding to the voice data; performing acoustic feature mapping on the voice representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the voice data; performing language rule learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information; performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data; performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value; and performing model training on the initial speech recognition model according to the language loss value to obtain a target speech recognition model.

[0087] For example, after obtaining a voiceprint accent enhancement characterization model with accent enhancement, the voiceprint accent enhancement characterization model can be used to provide support for the subsequent improvement of a speech recognition model with speech, and then the voiceprint accent enhancement characterization model can be used to capture speaker characteristics in speech data.

[0088] Exemplarily, based on the acoustic network of the initial speech recognition model, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a Transformer, the speech representation information obtained by the voiceprint accent enhancement representation model is input into the acoustic network, the speech representation information is processed by the acoustic network, and the target acoustic features are extracted.

[0089] For example, based on the language network of the initial speech recognition model, such as N-gram, or a more complex neural network language network (such as RNN, LSTM, Transformer, etc.), the text information is input into the language network, the language network is used to learn the language rules in the text data, and the target text features are extracted.

[0090] Exemplarily, according to the decoding layer of the initial speech recognition model, a decoder based on the Viterbi algorithm (Viterbi) or Beam Search can be used, so that the decoding layer is combined with the target acoustic features and the target text features to perform language prediction to generate the predicted text corresponding to the speech data.

[0091] Exemplarily, the loss function of the language loss layer of the initial speech recognition model is determined, such as cross entropy loss, CTC (Connectionist Temporal Classification) loss, etc. The predicted text is compared with the text information to calculate the language loss value. The prediction accuracy of the initial speech recognition model is evaluated by the language loss value, and error feedback is provided during the training process. Therefore, when the language loss value does not meet the requirements, the calculated language loss value is used to update the model parameters through back propagation, and the initial speech recognition model is iteratively trained using speech samples until the target speech recognition model is obtained.

[0092] Specifically, the target speech recognition model is based on the voiceprint accent enhancement representation model, thus providing strong support for the subsequent improvement of speech recognition accuracy for users with accents.

[0093] The embodiment of the present invention provides a training method, device, equipment and medium for a voiceprint recognition model, the method comprising: obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data; using a feature extraction layer of a voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; using a voiceprint embedding layer of a voiceprint recognition model to extract voiceprint representations from the initial sound features to obtain initial sound representation information; using an accent recognition layer of the voiceprint recognition model to perform accent recognition based on the initial sound representation information to obtain a first classification result; using an accent loss layer of the voiceprint recognition model to perform loss calculation based on the first classification result and the accent label to obtain a first loss value; using an identity recognition layer of the voiceprint recognition model to perform speaker recognition based on the initial sound representation information to obtain a second classification result; using a voiceprint loss layer of the voiceprint recognition model to perform loss calculation based on the second classification result and the speaker label to obtain a second loss value; and training the voiceprint recognition model based on the first loss value and the second loss value to obtain a target recognition model. This method adds a training loss for distinguishing different accents during the training process of the voiceprint recognition model, which can enhance the model's ability to learn accent differences during the training process. After the training is completed, the voiceprint recognition model can improve its ability to recognize accent differences while judging the speaker's voiceprint. When the voices of two people with similar voices but different accents are used for recognition, the voiceprint recognition model can comprehensively judge the voice characteristics and accent characteristics at the same time, thereby distinguishing that they are two different people, thereby improving the recognition accuracy and improving the defect that the existing algorithm cannot effectively utilize accent differences. This method adds a new judgment dimension to distinguish the identities of similar speakers. It also solves the problem that related technologies are difficult to distinguish between users with similar voices but different accents when facing users with different accents.

[0094] See also Figure 4 , Figure 4A training device 200 for a voiceprint recognition model provided in an embodiment of the present application includes a data acquisition module 201, a feature extraction module 202, a voiceprint embedding module 203, an accent recognition module 204, an accent loss module 205, an identity recognition module 206, a voiceprint loss module 207, and a model training module 208, wherein the data acquisition module 201 is used to obtain training data, wherein the training data includes initial audio data and an accent label and a speaker label corresponding to the initial audio data; the feature extraction module 202 is used to use the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; the voiceprint embedding module 203 is used to use the voiceprint embedding layer of the voiceprint recognition model to extract voiceprint representation from the initial sound features to obtain initial sound characterization information; an accent recognition module 204, used to use the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound characterization information to obtain a first classification result; an accent loss module 205, used to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; an identity recognition module 206, used to use the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound characterization information to obtain a second classification result; a voiceprint loss module 207, used to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; a model training module 208, used to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

[0095] In some implementations, during the process of training the voiceprint recognition model according to the first loss value and the second loss value to obtain the target recognition model, the model training module 208 performs:

[0096] Determine a hyperparameter, and determine a target loss value corresponding to the voiceprint recognition model by fusing the first loss value and the second loss value according to the hyperparameter;

[0097] The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

[0098] In some implementations, after obtaining the target recognition model, the voiceprint recognition model training device 200 further performs:

[0099] Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and a first model parameter corresponding to the first network structure;

[0100] Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and a second model parameter corresponding to the second network structure;

[0101] A voiceprint accent enhancement characterization model is determined according to the first network structure, the first model parameters, the second network structure and the second model parameters.

[0102] In some embodiments, after obtaining the voiceprint accent enhancement representation model, the voiceprint recognition model training device 200 further performs:

[0103] Obtain target audio data corresponding to the target user;

[0104] Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target sound characterization information corresponding to the target user;

[0105] The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

[0106] In some implementations, after determining the user identity corresponding to the target user according to the distance information, the voiceprint recognition model training device 200 further performs:

[0107] Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint accent enhancement representation model according to the target identity and the user identity;

[0108] Performing a model update on the target recognition model according to the accuracy rate to obtain an updated target recognition model;

[0109] The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

[0110] In some embodiments, the voiceprint recognition model training device 200 performs, during the process of updating the target recognition model according to the accuracy rate to obtain the updated target recognition model:

[0111] When the accuracy rate is lower than a preset value, determining incremental data based on data when the target identity and the user identity are inconsistent;

[0112] The target recognition model is incrementally trained according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

[0113] In some embodiments, after obtaining the voiceprint accent enhancement representation model, the voiceprint recognition model training device 200 further performs:

[0114] Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data;

[0115] Performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data;

[0116] Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data;

[0117] Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information;

[0118] Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data;

[0119] Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value;

[0120] The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

[0121] In some implementations, the voiceprint recognition model training device 200 may be applied to a terminal device.

[0122] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the training device 200 for the voiceprint recognition model described above can refer to the corresponding process in the aforementioned embodiment of the training method for the voiceprint recognition model, and will not be repeated here.

[0123] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention.

[0124] like Figure 5 As shown, the terminal device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected via a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0125] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0126] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0127] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the terminal device to which the embodiment of the present invention is applied. The specific server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] The processor is used to run the computer program stored in the memory, and implement any one of the voiceprint recognition model training methods provided by the embodiments of the present invention when executing the computer program.

[0129] In one embodiment, the processor is used to run a computer program stored in the memory, and implements the following steps when executing the computer program:

[0130] Obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data;

[0131] Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features;

[0132] Using the voiceprint embedding layer of the voiceprint recognition model to extract the voiceprint representation of the initial sound feature to obtain initial sound representation information;

[0133] Using the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result;

[0134] Using the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value;

[0135] Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result;

[0136] Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value;

[0137] The voiceprint recognition model is trained according to the first loss value and the second loss value to obtain a target recognition model.

[0138] In some implementations, during the process of training the voiceprint recognition model according to the first loss value and the second loss value to obtain the target recognition model, the processor 301 executes:

[0139] Determine a hyperparameter, and determine a target loss value corresponding to the voiceprint recognition model by fusing the first loss value and the second loss value according to the hyperparameter;

[0140] The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

[0141] In some implementations, after obtaining the target recognition model, the processor 301 further executes:

[0142] Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and a first model parameter corresponding to the first network structure;

[0143] Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and a second model parameter corresponding to the second network structure;

[0144] A voiceprint accent enhancement characterization model is determined according to the first network structure, the first model parameters, the second network structure and the second model parameters.

[0145] In some implementations, after obtaining the voiceprint accent enhancement representation model, the processor 301 further executes:

[0146] Obtain target audio data corresponding to the target user;

[0147] Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target sound characterization information corresponding to the target user;

[0148] The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

[0149] In some implementations, after determining the user identity corresponding to the target user according to the distance information, the processor 301 further executes:

[0150] Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint accent enhancement representation model according to the target identity and the user identity;

[0151] Performing a model update on the target recognition model according to the accuracy rate to obtain an updated target recognition model;

[0152] The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

[0153] In some implementations, the processor 301, in the process of updating the target recognition model according to the accuracy rate to obtain the updated target recognition model, executes:

[0154] When the accuracy rate is lower than a preset value, determining incremental data based on data when the target identity and the user identity are inconsistent;

[0155] The target recognition model is incrementally trained according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

[0156] In some implementations, after obtaining the voiceprint accent enhancement representation model, the processor 301 further executes:

[0157] Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data;

[0158] Performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data;

[0159] Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data;

[0160] Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information;

[0161] Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data;

[0162] Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value;

[0163] The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

[0164] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the embodiment of the training method of the aforementioned voiceprint recognition model, and will not be repeated here.

[0165] An embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any method for training a voiceprint recognition model provided in the description of the embodiment of the present invention.

[0166] The storage medium may be an internal storage unit of the terminal device in the aforementioned embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device.

[0167] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0168] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0169] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for training a voiceprint recognition model, characterized in that: The method comprises: Obtaining training data, the training data comprising initial audio data and an accent label and a speaker label corresponding to the initial audio data; Using the feature extraction layer of the voiceprint recognition model to extract sound features from the initial audio data to obtain initial sound features; Using the voiceprint embedding layer of the voiceprint recognition model to extract the voiceprint representation of the initial sound feature to obtain initial sound representation information; Using the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result; Using the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; Using the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result; Using the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; The voiceprint recognition model is trained according to the first loss value and the second loss value to obtain a target recognition model.

2. The method according to claim 1, characterized in that The performing model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model includes: Determine a hyperparameter, and determine a target loss value corresponding to the voiceprint recognition model by fusing the first loss value and the second loss value according to the hyperparameter; The target recognition model is obtained by performing model training on the voiceprint recognition model according to the target loss value.

3. The method according to claim 1, characterized in that After obtaining the target recognition model, the method further includes: Obtaining a first network structure corresponding to the feature extraction layer in the target recognition model and a first model parameter corresponding to the first network structure; Obtaining a second network structure corresponding to the voiceprint embedding layer in the target recognition model and a second model parameter corresponding to the second network structure; A voiceprint accent enhancement characterization model is determined according to the first network structure, the first model parameters, the second network structure and the second model parameters.

4. The method according to claim 3, characterized in that After obtaining the voiceprint accent enhancement representation model, the method further includes: Obtain target audio data corresponding to the target user; Performing voiceprint characterization processing on the target audio data according to the voiceprint accent enhancement characterization model to obtain target sound characterization information corresponding to the target user; The distance information between the target sound representation information and the preset sound representation information is calculated, and the user identity corresponding to the target user is determined according to the distance information.

5. The method according to claim 4, characterized in that After determining the user identity corresponding to the target user according to the distance information, the method further includes: Obtaining a target identity corresponding to the target user, and determining the accuracy of the voiceprint accent enhancement representation model according to the target identity and the user identity; Performing a model update on the target recognition model according to the accuracy rate to obtain an updated target recognition model; The voiceprint and accent enhancement representation model is updated according to the updated target recognition model to obtain the updated voiceprint and accent enhancement representation model.

6. The method according to claim 5, characterized in that The updating of the target recognition model according to the accuracy rate to obtain the updated target recognition model includes: When the accuracy rate is lower than a preset value, determining incremental data based on data when the target identity and the user identity are inconsistent; The target recognition model is incrementally trained according to the incremental data, and then the target recognition model is updated to obtain the updated target recognition model.

7. The method according to claim 3, characterized in that After obtaining the voiceprint accent enhancement representation model, the method further includes: Determine a speech sample corresponding to the initial speech recognition model training, wherein the speech sample includes speech data and text information corresponding to the speech data; Performing voiceprint characterization processing on the speech data according to the voiceprint accent enhancement characterization model to obtain speech characterization information corresponding to the speech data; Performing acoustic feature mapping on the speech representation information according to the acoustic network of the initial speech recognition model to obtain target acoustic features corresponding to the speech data; Performing language regularity learning on the text information according to the language network of the initial speech recognition model to obtain target text features corresponding to the text information; Performing language prediction using the target acoustic features and the target text features according to the decoding layer of the initial speech recognition model to obtain predicted text corresponding to the speech data; Performing loss calculation using the predicted text and the text information according to the language loss layer of the initial speech recognition model to obtain a language loss value; The initial speech recognition model is trained according to the language loss value to obtain a target speech recognition model.

8. A training device for a voiceprint recognition model, characterized in that: include: A data acquisition module, used to obtain training data, wherein the training data includes initial audio data and an accent label and a speaker label corresponding to the initial audio data; A feature extraction module, used to extract sound features from the initial audio data using the feature extraction layer of the voiceprint recognition model to obtain initial sound features; A voiceprint embedding module, configured to extract voiceprint representation of the initial sound feature using the voiceprint embedding layer of the voiceprint recognition model to obtain initial sound representation information; an accent recognition module, configured to use the accent recognition layer of the voiceprint recognition model to perform accent recognition according to the initial sound representation information to obtain a first classification result; an accent loss module, configured to use the accent loss layer of the voiceprint recognition model to perform loss calculation according to the first classification result and the accent label to obtain a first loss value; An identity recognition module, configured to use the identity recognition layer of the voiceprint recognition model to perform speaker recognition according to the initial sound representation information to obtain a second classification result; A voiceprint loss module, configured to use the voiceprint loss layer of the voiceprint recognition model to perform loss calculation according to the second classification result and the speaker label to obtain a second loss value; A model training module is used to perform model training on the voiceprint recognition model according to the first loss value and the second loss value to obtain a target recognition model.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and implement the training method of the voiceprint recognition model according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium for computer storage, characterized in that: The computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the voiceprint recognition model training method described in any one of claims 1 to 7.

Images