Voiceprint recognition model training method and device, voiceprint recognition method and device

Abstract

The present disclosure provides a method and device for training a voiceprint recognition model, which relates to technical fields such as speech technology and deep learning. The specific implementation scheme is: obtaining a feature sample set; obtaining a voiceprint recognition network, which includes: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in the input information, and the basic network is used to predict the speaker features; the mutual adversarial network is used to distinguish speaker features from non-speaker features; the selected recognition vector features are input into the feature extractor to obtain speaker features and non-speaker features corresponding to the selected recognition vector features; the speaker features and the non-speaker features are input into the basic network, and the speaker features and the non-speaker features are input into the mutual adversarial network, and the loss value of the voiceprint recognition network is calculated; based on the loss value and the voiceprint recognition network, a voiceprint recognition model is obtained. This implementation improves the accuracy of voiceprint recognition.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to technical fields such as speech technology and deep learning, and in particular to a voiceprint recognition model training method and device, a voiceprint recognition method and device, an electronic device, a computer-readable medium, and a computer program product. Background Art

[0002] In actual voiceprint system applications, there will be interference from multiple phonemes such as the speaker himself and the environment, which will have a certain impact on the accuracy of voiceprint feature extraction. How to eliminate these influences and extract the speaker's characteristics more accurately has become a hot research field in voiceprint in recent years. Summary of the invention

[0003] The present disclosure provides a voiceprint recognition model training method and device, a voiceprint recognition method and device, an electronic device, a computer-readable medium, and a computer program product.

[0004] According to a first aspect, a method for training a voiceprint recognition model is provided, the method comprising: obtaining a feature sample set including at least one recognition vector feature; obtaining a pre-established voiceprint recognition network, the voiceprint recognition network comprising: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in input information, the basic network is used to predict speaker features; the mutual adversarial network is used to distinguish speaker features from non-speaker features; performing the following training steps: inputting the recognition vector features selected from the feature sample set into the feature extractor to obtain speaker features and non-speaker features corresponding to the selected recognition vector features; inputting the speaker features and the non-speaker features into the basic network, and inputting the speaker features and the non-speaker features into the mutual adversarial network, and calculating the loss value of the voiceprint recognition network; obtaining a voiceprint recognition model based on the loss value and the voiceprint recognition network.

[0005] According to the second aspect, a voiceprint recognition method is provided, the method comprising: obtaining recognition vector features of a speech to be recognized; inputting the recognition vector features of the speech to be recognized into a voiceprint recognition model generated by the method described in any implementation of the first aspect, and outputting a voiceprint recognition result of the speech to be recognized.

[0006] According to a third aspect, a voiceprint recognition model training device is provided, the device comprising: a sample acquisition unit, configured to acquire a feature sample set including at least one recognition vector feature; a network acquisition unit, configured to acquire a pre-established voiceprint recognition network, the voiceprint recognition network comprising: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in input information, the basic network is used to predict speaker features; the mutual adversarial network is used to distinguish speaker features from non-speaker features; the feature acquisition unit is configured to input the recognition vector features selected from the feature sample set into the feature extractor, and obtain the speaker features and non-speaker features corresponding to the selected recognition vector features; a calculation unit is configured to input the speaker features and the non-speaker features into the basic network, and input the speaker features and the non-speaker features into the mutual adversarial network, and calculate the loss value of the voiceprint recognition network; an output unit is configured to obtain a voiceprint recognition model based on the loss value and the voiceprint recognition network.

[0007] According to the fourth aspect, a voiceprint recognition device is provided, which includes: an acquisition unit, configured to acquire recognition vector features of a speech to be recognized; a recognition unit, configured to input the recognition vector features of the speech to be recognized into a voiceprint recognition model generated by the device described in any implementation method of the third aspect, and output a voiceprint recognition result of the speech to be recognized.

[0008] According to the fifth aspect, an electronic device is provided, which includes: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation of the first aspect or the second aspect.

[0009] According to a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described in any implementation of the first aspect or the second aspect.

[0010] According to a seventh aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method described in any implementation of the first aspect or the second aspect.

[0011] The voiceprint recognition model training method and device provided by the embodiments of the present disclosure first obtain a feature sample set including at least one recognition vector feature; secondly, obtain a pre-established voiceprint recognition network, which includes: a feature extractor, a basic network and a mutual adversarial network; thirdly, input the recognition vector feature selected from the feature sample set into the feature extractor to obtain the speaker feature and non-speaker feature corresponding to the selected recognition vector feature; thirdly, input the speaker feature and the non-speaker feature into the basic network, and input the speaker feature and the non-speaker feature into the mutual adversarial network to calculate the loss value of the voiceprint recognition network; finally, based on the loss value and the voiceprint recognition network, obtain the voiceprint recognition model. Therefore, by constructing a voiceprint recognition network including a feature extractor, a basic network, and a mutual adversarial network, the feature extractor can be used to extract speaker features and non-speaker features, the basic network can be used to predict speaker features, and the mutual adversarial network can be used to distinguish speaker features and non-speaker features, so that speaker features and non-speaker features can be represented separately, and speaker features and non-speaker features can be differentiated as much as possible in the voiceprint recognition network training until the two are balanced, so as to achieve the purpose of purer speaker features, thereby improving the accuracy of voiceprint recognition by the voiceprint recognition model.

[0012] The voiceprint recognition method and device provided by the embodiments of the present disclosure obtain the recognition vector features of the speech to be recognized; input the recognition vector features of the speech to be recognized into the voiceprint recognition model generated by the voiceprint recognition model training method of the present embodiment, and output the voiceprint recognition result of the speech to be recognized. Thus, the voiceprint recognition model generated by the feature extractor, the basic network and the mutual resistance network can improve the accuracy of the voiceprint recognition result of the speech to be recognized.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0015] Figure 1 is a flow chart of an embodiment of a method for training a voiceprint recognition model according to the present disclosure;

[0016] Figure 2 This is a schematic diagram of a structure for training a voiceprint recognition network in an embodiment of the present disclosure;

[0017] Figure 3 is a flow chart of an embodiment of a voiceprint recognition method according to the present disclosure;

[0018] Figure 4is a structural schematic diagram of an embodiment of a voiceprint recognition model training device according to the present disclosure;

[0019] Figure 5 is a structural schematic diagram of an embodiment of a voiceprint recognition device according to the present disclosure;

[0020] Figure 6 It is a block diagram of an electronic device used to implement the voiceprint recognition model training method or voiceprint recognition method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] Traditional voiceprint recognition modeling is generally divided into three modules:

[0023] Acoustic feature extraction module: Acoustic features are a kind of low-level features, which are obtained by processing audio data through traditional digital signal processing technology. Generally, MFCC (Mel-frequency cepstral coefficients), PLP (Perceptual Linear Predictive), Fbank (Filter Bank) and other methods are used for processing.

[0024] Identification vector feature extraction module: The identification vector feature is a high-level feature. The methods for extracting identification vector features generally include unsupervised GMM (Gaussian Mix Models), such as the collection of identification vector i-vector features; supervised deep learning-based DNN (Deep Neural Networks), CNN (Convolutional Neural Network), etc., such as the collection of identification vector x-vector features.

[0025] Backend classification models: usually Cos (cosine), LDA (linear discriminant analysis), PLDA (linear discriminant analysis with probability).

[0026] Generally, in order to improve the recognition effect of voiceprints, we can start from one or more modules of the above three models, optimize various strategies, and try to improve the accuracy of voiceprint recognition.

[0027] The recognition vector features extracted by the recognition vector feature extraction module contain some redundant information (such as speech speed, volume, emotion, channel, device, etc.), which affects the purity of the speaker's characteristics (such as gender, accent, physiological structure, pronunciation habits, etc.). Therefore, this application proposes the following embodiment for removing redundant information in the recognition vector features to improve the accuracy of voiceprint recognition.

[0028] Figure 1 A process 100 according to an embodiment of a voiceprint recognition model training method disclosed in the present invention is shown. The voiceprint recognition model training method includes the following steps:

[0029] Step 101: Acquire a feature sample set including at least one recognition vector feature.

[0030] In this embodiment, the execution subject on which the voiceprint recognition model training method runs can obtain the sample set in a variety of ways. For example, the execution subject can obtain the feature sample set stored in the database server through a wired connection or a wireless connection. For another example, the user can obtain the recognition vector feature by processing the voice data through the terminal. In this way, the execution subject can receive the recognition vector feature collected by the terminal and store such recognition vector feature locally, thereby generating a feature sample set.

[0031] Here, the feature sample set may include at least one recognition vector feature. Recognition vector features are high-end features relative to acoustic features. The high-end features are features extracted from acoustic features using some feature extraction models and can identify speaker information, where speaker information is used to distinguish the speaker's identity. Speaker information may include: gender, accent, physiological structure, pronunciation habits, etc. In order to train the voiceprint model, in this embodiment, the recognition vector features may include original speaker features and annotation information annotated on the original speaker features.

[0032] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of speaker information and original speaker features are carried out after the speaker's authorization and comply with relevant laws and regulations.

[0033] In this embodiment, the original speaker features may be original speaker features extracted by a speaker feature extraction method. The original speaker features, such as i-vector (identifying vector) features, include some redundant information (such as speech speed, volume, emotion, and channels of different pickup devices). The above-mentioned annotation information annotated on the original speaker features is used to indicate the identity of the speaker, such as Zhang San, Li Si, etc. The voiceprint recognition model provided by the present disclosure can extract the speaker features in the original speaker features from the redundant information to ensure the purity of the speaker features, that is, the identification vector features represent the speaker features as much as possible.

[0034] In this embodiment, the feature sample set can be a feature sample set obtained by extracting feature sequences of multiple audio data through a pre-trained feature extraction model. The feature sample set can also use multiple audio data to train a feature extraction network, and use the trained feature extraction model to extract identification features of the audio data to obtain a feature sample set.

[0035] In some optional implementations of the present embodiment, the above-mentioned acquisition of a feature sample set including at least one recognition vector feature includes: collecting at least one audio data; for each audio data in the at least one audio data, framing the audio data to obtain multiple audio frames; performing acoustic feature extraction on each audio frame in the multiple audio frames to obtain a feature sequence, wherein the feature sequence is used to characterize the acoustic features of multiple audio frames of the audio data; performing recognition feature extraction on the feature sequence to obtain a recognition vector feature of the audio data.

[0036] In this optional implementation, a trained feature extraction model may be used to extract identification features from the feature sequence, and the feature extraction model is used to characterize the corresponding relationship between the feature sequence and the identification vector feature.

[0037] In this optional implementation, when the identification vector feature is i-vector, the traditional GMM model can be used as the feature extraction model; when the identification vector feature is x-vector, the supervised neural network model can be used as the feature extraction model.

[0038] In this optional implementation, a trained feature extraction model is used to extract identification vector features from the feature sequence of acoustic features, without the need to label the speaker identity of the audio data, and the implementation is simple and convenient.

[0039] In another optional implementation of the present embodiment, the above-mentioned acquisition of a feature sample set including at least one recognition vector feature includes: collecting multiple audio data; for each audio data in the multiple audio data, performing speaker labeling on the audio data to obtain speaker labeling information; framing the audio data to obtain multiple audio frames of the audio data; performing acoustic feature extraction on each audio frame in the multiple audio frames to obtain a feature sequence corresponding to the audio data, wherein the feature sequence is used to characterize the acoustic features of multiple audio frames of the audio data; using the feature sequence corresponding to each audio data and the speaker labeling information of each audio data to train a pre-constructed feature extraction network to obtain a feature extraction model, the feature extraction network being used to characterize the correspondence between the feature sequence and the recognition vector feature; and inputting the feature sequence corresponding to each audio data in the multiple audio data into the feature extraction model in sequence to obtain a feature sample set including multiple recognition vector features.

[0040] In this optional implementation, the process of obtaining the feature sample set includes: collecting a certain amount of audio data (such as 1000 people, 10 audio data for each person), annotating the audio with the speaker identity (such as Zhang San, Li Si), dividing the audio data into frames, and then extracting features from each frame to obtain a feature sequence such as x1, x2...x T The feature sequence is input into the x-vector network, and the cross entropy loss function is used to calculate the x-vector network loss. Then, the x-vector network parameters are updated in reverse, and multiple rounds of iterations are repeated until the x-vector network converges to obtain a feature extraction model. The feature extraction model is used to extract the recognition vector features of each audio data to obtain a training sample set.

[0041] In this optional implementation, by labeling the speaker identity of the audio data and using the labeled audio data to train the feature extraction network, it can be ensured that the recognition vector features output in the feature extraction model are more consistent with the current audio data, thereby improving the accuracy of the feature sample set.

[0042] Optionally, the above-mentioned feature sample set can also include the following three steps: Step 1: Collect a certain amount of audio data (such as 1000 people, 10 audio data for each person), annotate the audio with the speaker identity (such as Zhang San, Li Si), and then preprocess the annotated audio data, the preprocessing includes noise removal (including environmental noise, busy tone, ringtone, etc.), data enhancement (aliasing echo, change rate, random masking in time domain and frequency domain); Step 2: Perform voice activity detection, framing and acoustic feature extraction on the audio data, such as MFCC, Fbank, PLP, etc., and regularize the feature mean. Step 3: Input into a multi-layer TDNN (Time Delay Neural Networks), use the cross entropy loss function to calculate the TDNN network loss, and then reversely update the TDNN network parameters, iterate multiple times until the TDNN network converges, obtain a feature extraction model, and use the feature extraction model to extract the recognition vector features of each audio data to obtain a training sample set.

[0043] Step 102: Acquire a pre-established voiceprint recognition network.

[0044] In this embodiment, the voiceprint recognition network is a network used to perform speaker feature recognition on the recognition vector features. Purer speaker features can be obtained through the voiceprint recognition network, thereby ensuring the accuracy of speaker feature recognition by the back-end classification model.

[0045] Among them, the voiceprint recognition network includes: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in the input information, the basic network is used to predict the speaker features; the mutual adversarial network is used to distinguish speaker features from non-speaker features.

[0046] In this embodiment, the speaker feature is information reflecting the speaker's identity, such as a pure x-vector vector used only to characterize the speaker; the non-speaker feature is a feature other than the speaker feature in the identification vector feature, such as speech rate feature, volume feature, emotion feature, etc.

[0047] In this embodiment, the input information includes the annotation information, and the speaker identity corresponding to the speaker feature is determined by the annotation information. When the voiceprint recognition network is iteratively trained, the input information is the recognition vector feature selected from the feature sample set, and the recognition vector feature is annotated with the annotation information. Figure 2 As shown, the input information is the identification vector feature X input to the feature extractor Enc. The feature extractor extracts the feature of the identification vector X to obtain the speaker feature h1 and the non-speaker feature h2.

[0048] In this embodiment, the feature extractor maps the input information into a high-dimensional feature space, the basic network predicts the speaker corresponding to the speaker feature in the input information, and the mutual adversarial network adopts an adversarial approach to make the speaker features and non-speaker features as different as possible and maximize the difference between the two to ensure that the two have the greatest differentiation. The mutual adversarial network can divide the speaker features and non-speaker features output by the feature extractor to ensure the independence of the speaker features and non-speaker features.

[0049] In this embodiment, the basic network is a network that simultaneously distinguishes speaker features and non-speaker features, and the mutual adversarial network is a network that distinguishes speaker features from non-speaker features as much as possible. When the loss value obtained by the voiceprint recognition network through the loss function converges, or the number of training iterations of the voiceprint recognition network reaches a preset number, it is determined that the training of the voiceprint recognition network is completed.

[0050] In step 103, the recognition vector features selected from the feature sample set are input into a feature extractor to obtain speaker features and non-speaker features corresponding to the selected recognition vector features.

[0051] In this embodiment, the execution subject may select the recognition vector feature from the feature sample set obtained in step 101, and perform the training steps from step 103 to step 105. The selection method and the number of the recognition vector features are not limited in this application. For example, in one iterative training, one recognition vector feature may be randomly selected, or multiple recognition vector features may be selected from the feature sample set in one iterative training. The loss value of the voiceprint recognition network is calculated by the original speaker feature in each selected recognition vector feature and the annotation information annotated on the original speaker feature, and the parameters of the voiceprint recognition network are adjusted.

[0052] In this embodiment, the main function of the feature extractor is to map the input recognition vector feature to a high-dimensional feature space to obtain a high-dimensional feature. The feature extractor can be an encoder. For example, the feature extractor is composed of two layers of DNN, each layer of DNN is 512-dimensional. Figure 2 As shown, the feature extractor Enc extracts features from the recognition vector feature X to obtain speaker features h1 and non-speaker features h2, where speaker feature h1 is input into the basic network to predict the corresponding speaker probability y, so the speaker feature h1 carries more speaker information; and the non-speaker feature h2 is input into the basic network to reconstruct the input information in order to maximize the restoration of the input information.

[0053] Step 104: input the speaker feature and the non-speaker feature into a basic network, and input the speaker feature and the non-speaker feature into an adversarial network to calculate a loss value of the voiceprint recognition network.

[0054] In this embodiment, the basic network is used to predict the speaker feature and the speaker feature corresponding to the speaker feature in the non-speaker feature to obtain a speaker prediction result.

[0055] Optionally, the basic network can also reconstruct the speaker feature and the non-speaker feature to obtain a reconstruction result corresponding to the selected recognition vector feature. In this embodiment, the speaker feature and the non-speaker feature are input into the voiceprint recognition network to obtain the speaker prediction result output by the voiceprint recognition network, the reconstruction result after reconstructing the recognition vector feature, the predicted speaker feature, the predicted non-speaker feature and other results. In each iterative training, the loss value of the voiceprint recognition network is calculated once.

[0056] In some optional implementations of the present embodiment, the above-mentioned inputting the speaker feature and the non-speaker feature into the basic network and the mutual adversarial network respectively, and calculating the loss value of the voiceprint recognition network, includes: inputting the speaker feature and the non-speaker feature into the basic network, and calculating the loss value of the basic network; inputting the speaker feature and the non-speaker feature into the mutual adversarial network, and calculating the loss value of the mutual adversarial network; based on the loss value of the basic network and the loss value of the mutual adversarial network, calculating the loss value of the voiceprint recognition network.

[0057] In this optional implementation, the loss value of the voiceprint recognition network is calculated based on the loss value of the basic network and the loss value of the mutual resistance network, including: adding the loss value of the basic network and the loss value of the mutual resistance network to obtain the loss value of the voiceprint recognition network.

[0058] Optionally, the above-mentioned calculation of the loss value of the voiceprint recognition network based on the loss value of the basic network and the loss value of the mutual resistance network may also include: determining the weight value of the basic network and the weight value of the mutual resistance network, multiplying the weight value of the basic network by the loss value of the basic network plus the product of the loss value of the mutual resistance network and the weight value of the mutual resistance network, to obtain the loss value of the voiceprint recognition network. The weight value of the basic network and the weight value of the mutual resistance network can be set according to training requirements.

[0059] In this optional implementation, the voiceprint recognition network represents the speaker characteristics and non-speaker characteristics separately, and increases the distinguishability of the two by introducing a mutual adversarial network. In this way, some redundant information contained in the recognition vector, such as speech speed, volume, emotion, channel (different sound pickup devices), etc., is decomposed. In this way, the speaker characteristics (gender, accent, physiological structure, pronunciation habits, etc.) obtained are purer, thereby improving the overall accuracy of voiceprint recognition.

[0060] In some optional implementations of the present embodiment, the basic network includes: an identity prediction module and a reconstruction module, the identity prediction module is used to perform speaker prediction on the speaker feature, and the reconstruction module is used to reconstruct the selected recognition vector feature of the non-speaker feature; the above-mentioned inputting the speaker feature and the non-speaker feature into the basic network and calculating the loss value of the basic network includes: inputting the speaker feature into the identity prediction module to obtain the speaker probability output by the identity prediction module, and calculating the identity loss value of the identity prediction module; removing a part of the proportion of features in the speaker feature to obtain the remaining features; inputting the remaining features and the non-speaker feature in series into the reconstruction module to obtain the reconstructed features output by the reconstruction module, and calculating the reconstruction loss value of the reconstruction module; based on the identity loss value and the reconstruction loss value, calculating the loss value of the basic network.

[0061] In this optional implementation, the cross entropy loss function can be used to calculate the identity loss value of the identity prediction module, which has a small amount of calculation, is convenient and simple to calculate. The mean square error loss function can also be used to calculate the reconstruction loss value of the reconstruction module, and the mean square error loss function can effectively evaluate the degree of data change.

[0062] In this optional implementation, the identity prediction module uses a prediction network to predict which speaker the speaker feature belongs to. When training the identity prediction module, the label information on the original speaker feature in the recognition vector feature is used to provide the identity prediction module with reliable prior information to effectively calculate the identity loss value of the identity prediction module. Figure 2 In , the identity prediction module Pre predicts the speaker feature h1 and outputs the probability y that the speaker feature h1 belongs to different speakers.

[0063] In this optional implementation, the reconstruction module may be a decoding network for reconstructing the original input information.

[0064] In order to increase the robustness of the model, the basic network can also include: a discard module, which is used to discard features of a set proportion, such as Figure 2 As shown, the speaker feature h1 is input into the discarding module Dropout, after a set proportion of speaker features are discarded in the discarding module Dropout, it is connected in series with the non-speaker feature h2 and input into the reconstruction module Dec. The reconstruction module Dec attempts to restore the selected recognition vector feature X to obtain the reconstructed feature X'.

[0065] Here, the speaker feature h1 suffers a certain amount of information loss or interference from noise after the Dropout module is discarded, so in the reconstruction process of the reconstruction module Dec, the non-speaker feature h2 is forced to carry more information different from the speaker feature h1 to restore the selected recognition vector feature X.

[0066] Optionally, the non-speaker feature h2 may be input into a discarding module Dropout, after a set proportion of non-speaker features are discarded in the discarding module Dropout, connected in series with the speaker feature h1, and input into a reconstruction module Dec to obtain a reconstructed feature X' output by the reconstruction module Dec.

[0067] In this optional implementation, a portion of the speaker features is removed to obtain the remaining features; the remaining features are connected in series with the non-speaker features and input into the reconstruction module, which can increase the robustness of the basic network during the training process and ensure the reliability of the results obtained by the basic network.

[0068] In some optional implementations of the present embodiment, the above-mentioned mutual adversarial network includes: a non-speaker prediction module and a speaker prediction module, the non-speaker prediction module is used to perform non-speaker prediction on speaker features, and the speaker prediction module is used to perform speaker prediction on non-speaker features, the speaker features and the non-speaker features are input into the mutual adversarial network, and the loss value of the mutual adversarial network is calculated, including: inputting the speaker features into the non-speaker prediction module to obtain the non-speaker features output by the non-speaker prediction module, and calculating the loss value of the non-speaker prediction module; inputting the non-speaker features into the speaker prediction module to obtain the speaker features output by the speaker prediction module, and calculating the loss value of the speaker prediction module; based on the loss value of the non-speaker prediction module and the loss value of the speaker prediction module, the loss value of the mutual adversarial network is calculated.

[0069] In this optional implementation, the mutual adversarial network includes: a non-speaker prediction module and a speaker prediction module, and both the non-speaker prediction module and the speaker prediction module can be composed of two layers of DNN, and each layer of DNN is 128 dimensions. Figure 2 As shown, the input of the non-speaker prediction module Dis1 is the speaker feature h1, and the output is the prediction result h2' for the non-speaker. The input of the speaker prediction module Dis2 is the non-speaker feature h2, and the output is the prediction result h1' for the speaker.

[0070] The mutual adversarial network is not used for reconstruction, but to use an adversarial approach to make the speaker feature h1 and the non-speaker feature h2 as different as possible. During the training process, the non-speaker prediction module Dis1 adopts the mean square error loss function and maximizes the loss value of the non-speaker prediction module; during the training process, the speaker prediction module Dis2 adopts the mean square error loss function and maximizes the loss value of the speaker prediction module.

[0071] In this embodiment, during the training process of the voiceprint recognition network, the loss value of the non-speaker prediction module and the loss value of the speaker prediction module can be directly added to obtain the loss value of the mutual adversarial network.

[0072] Optionally, during the training process of the voiceprint recognition network, weight values ​​can be assigned to the loss value of the non-speaker prediction module and the loss value of the speaker prediction module respectively, and the loss value of the mutual adversarial network can be obtained based on the weight value of the loss value of the non-speaker prediction module, the weight value of the loss value of the speaker prediction module, the loss value of the non-speaker prediction module, and the loss value of the speaker prediction module.

[0073] In this optional implementation, the mean square error loss function can be used to calculate the loss value of the non-speaker prediction module, and the mean square error loss function can be used to calculate the value of the speaker prediction module. When calculating the loss value of the entire voiceprint recognition network, the loss value of the voiceprint recognition network is obtained by maximizing the loss value of the mutual adversarial network and minimizing the loss value of the basic network.

[0074] In this optional implementation, by maximizing the loss values ​​of the non-speaker prediction model and the speaker prediction model, speaker features and non-speaker features can be effectively distinguished, which is beneficial to the effective decomposition of speaker features.

[0075] Step 105: Obtain a voiceprint recognition model based on the loss value and the voiceprint recognition network.

[0076] In this embodiment, the loss value of the voiceprint recognition network can be used to detect whether the voiceprint recognition network meets the training completion conditions. After the voiceprint recognition network meets the training completion conditions, the voiceprint recognition model obtained by the trained voiceprint recognition network is an effective and reliable model that can extract pure speaker features.

[0077] In this embodiment, the voiceprint recognition model obtained by the trained voiceprint recognition network includes: using the feature extractor in the trained voiceprint recognition network as the voiceprint recognition model.

[0078] Optionally, the voiceprint recognition model obtained by the trained voiceprint recognition network includes: using the feature extractor and the basic network in the trained voiceprint recognition network together as the voiceprint recognition model.

[0079] In this embodiment, the training completion condition includes at least one of the following: the number of training iterations of the voiceprint recognition network reaches a predetermined iteration threshold, and the loss value of the voiceprint recognition network is less than a predetermined loss value threshold. The predetermined iteration threshold is an empirical value obtained based on the loss value of the voiceprint recognition network. For example, the predetermined iteration threshold of the voiceprint recognition network is 5,000 times. The predetermined loss value threshold of the voiceprint recognition network is 0.05.

[0080] Optionally, in this embodiment, in response to the voiceprint recognition network not meeting the training completion condition, the relevant parameters in the voiceprint recognition network are adjusted so that the loss value of the voiceprint recognition network converges, and the above training steps 103-105 are continued to be executed based on the adjusted voiceprint recognition network.

[0081] See also Figure 2 As shown, each recognition vector feature X is input into the feature extractor Enc to obtain the speaker feature h1 and the non-speaker feature h2. The speaker feature h1 is input into the identity prediction module Pre to predict the corresponding speaker probability y, and the identity loss value is calculated with the annotation information in the recognition vector feature X. The speaker feature h1 is input into the reconstruction module Dec in series through the discarding module Dropout and the non-speaker feature h2 to reconstruct the input feature and calculate the reconstruction loss value with the recognition vector feature X. Based on the identity loss value and the reconstruction loss value, the loss value of the basic network is obtained.

[0082] Furthermore, the speaker feature h1 is input into the non-speaker prediction module Dis1, and the non-speaker feature h2 is input into the speaker prediction module Dis2, and the prediction results h2' and h1' for the non-speaker are predicted respectively; the loss value of the non-speaker prediction module Dis1 is calculated by the non-speaker prediction result h2' and the non-speaker feature h2, and the loss value of the speaker prediction module Dis2 is calculated by the speaker prediction result h1' and the speaker feature h1; based on the loss value of the non-speaker prediction module Dis1 and the loss value of the speaker prediction module Dis2, the loss value of the mutual adversarial network is calculated by maximizing the loss value of the basic network and maximizing the loss value of the mutual adversarial network; under the guidance of the objective function of the entire voiceprint recognition network, the loss is reversely transferred according to the SGD (Stochastic Gradient Descent) criterion, and the parameters of the voiceprint recognition network are updated; in this way, multiple rounds of iterations are repeated until the entire voiceprint recognition network converges, and a stable voiceprint recognition model can be obtained.

[0083] In this optional implementation, when the voiceprint recognition network does not meet the training completion conditions, adjusting the relevant parameters of the voiceprint recognition network is helpful to help the loss value of the voiceprint recognition network converge.

[0084] In this embodiment, if the training is not completed, the parameters of the voiceprint recognition network are adjusted to make the loss value of the voiceprint recognition network converge. Specifically, the parameters of the basic network can be kept unchanged, and steps 203 to 205 can be repeatedly performed to adjust the parameters of the corresponding mutual resistance network so that the loss value of the mutual resistance network gradually increases until it stabilizes. Then, the parameters of each mutual resistance network are kept unchanged, and steps 203 to 205 are repeatedly performed to adjust the parameters of the basic network so that the loss value of the basic network gradually decreases until it stabilizes. The parameters of the mutual resistance network and the parameters of the basic network are alternately trained until the loss value converges.

[0085] Optionally, during each iteration, the parameters of the mutual adversarial network and the parameters of the basic network can be adjusted simultaneously to ensure that the loss value of the mutual adversarial network gradually increases until it stabilizes, and the loss value of the basic network gradually decreases until it stabilizes. When the loss value changes of the mutual adversarial network and the basic network are small, it is determined that the voiceprint recognition network training is completed.

[0086] In this embodiment, when testing the voiceprint recognition model, the audio data may be firstly feature extracted to obtain a discriminant vector feature X, and then the speaker feature may be obtained through the voiceprint recognition model. The obtained speaker feature may be compared with the speaker feature of a known speaker for similarity to obtain the tested speaker.

[0087] The voiceprint recognition model training method and device provided by the embodiments of the present disclosure first obtain a feature sample set including at least one recognition vector feature; secondly, obtain a pre-established voiceprint recognition network, which includes: a feature extractor, a basic network and a mutual adversarial network; thirdly, input the recognition vector feature selected from the feature sample set into the feature extractor to obtain the speaker feature and non-speaker feature corresponding to the selected recognition vector feature; thirdly, input the speaker feature and the non-speaker feature into the basic network, and input the speaker feature and the non-speaker feature into the mutual adversarial network to calculate the loss value of the voiceprint recognition network; finally, based on the loss value and the voiceprint recognition network, obtain the voiceprint recognition model. Therefore, by constructing a voiceprint recognition network including a feature extractor, a basic network, and a mutual adversarial network, the feature extractor can be used to extract speaker features and non-speaker features, the basic network can be used to predict speaker features, and the mutual adversarial network can be used to distinguish speaker features and non-speaker features, so that speaker features and non-speaker features can be represented separately, and speaker features and non-speaker features can be differentiated as much as possible in the voiceprint recognition network training until the two are balanced, so as to achieve the purpose of purer speaker features, thereby improving the accuracy of voiceprint recognition by the voiceprint recognition model.

[0088] Furthermore, based on the voiceprint recognition model training method provided in the above embodiment, the present disclosure also provides an embodiment of a voiceprint recognition method. The voiceprint recognition method of the present disclosure combines artificial intelligence fields such as speech technology and deep learning.

[0089] See also Figure 3 , shows a process 300 according to an embodiment of the voiceprint recognition method disclosed in the present invention. The voiceprint recognition method provided in this embodiment includes the following steps:

[0090] Step 301: Obtain recognition vector features of the speech to be recognized.

[0091] In this embodiment, the execution subject of the voiceprint recognition method can obtain the recognition vector features of the speech to be recognized in a variety of ways. For example, the execution subject can obtain the recognition vector features of the speech to be recognized stored in the database server through a wired connection or a wireless connection. For another example, the execution subject can also receive in real time the recognition vector features of the speech to be recognized obtained after the terminal or other device extracts the features of the audio data to be recognized.

[0092] In this embodiment, the recognition vector features of the speech to be recognized obtained are recognition vector features of an undetermined speaker. The recognition vector features of the speech to be recognized may be high-level features extracted by a feature extraction module (such as GMM, DNN). Specifically, the recognition vector features of the speech to be recognized may be i-vector features or x-vector features corresponding to the speech to be recognized, or a variant of i-vector features and x-vector features, such as d-vector.

[0093] Step 302: input the recognition vector feature of the speech to be recognized into the voiceprint recognition model, and output the voiceprint recognition result of the speech to be recognized.

[0094] In this embodiment, the execution subject can input the recognition vector features of the speech to be recognized obtained in step 301 into the voiceprint recognition model, so as to obtain the voiceprint recognition result of the speech to be recognized. It should be noted that the voiceprint recognition result is the result of performing voiceprint recognition on the recognition vector features of the speech to be recognized. Based on the structure of the voiceprint recognition model, it can be known that the obtained voiceprint recognition result can more accurately represent the speaker characteristics of the speech to be recognized.

[0095] In this embodiment, the voiceprint recognition model can be adopted as described above. Figure 1 The specific training process can be found in Figure 1 The relevant description of the embodiments will not be repeated here.

[0096] In this embodiment, the speaker features in the voiceprint recognition result can be compared with the speaker features of known speakers for similarity, so as to obtain the speaker of the speech to be recognized.

[0097] The voiceprint recognition method provided by the embodiment of the present disclosure obtains the recognition vector features of the speech to be recognized; inputs the recognition vector features of the speech to be recognized into the voiceprint recognition model generated by the voiceprint recognition model training method of the present embodiment, and outputs the voiceprint recognition result of the speech to be recognized. Thus, the voiceprint recognition model generated by the feature extractor, the basic network and the mutual resistance network can improve the accuracy of the voiceprint recognition result of the speech to be recognized.

[0098] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a voiceprint recognition model training device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0099] like Figure 4 As shown, the voiceprint recognition model training device 400 provided in this embodiment includes: a sample acquisition unit 401, a network acquisition unit 402, a feature acquisition unit 403, a calculation unit 404, and an output unit 405. Among them, the above-mentioned sample acquisition unit 401 can be configured to be configured to acquire a feature sample set including at least one recognition vector feature. The above-mentioned network acquisition unit 402 can be configured to be configured to acquire a pre-established voiceprint recognition network, and the voiceprint recognition network includes: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in the input information, and the basic network is used to predict the speaker features; the mutual adversarial network is used to distinguish between speaker features and non-speaker features. The above-mentioned feature acquisition unit 403 can be configured to input the recognition vector features selected from the feature sample set into the feature extractor to obtain the speaker features and non-speaker features corresponding to the selected recognition vector features. The calculation unit 404 may be configured to input the speaker feature and the non-speaker feature into the basic network, and input the speaker feature and the non-speaker feature into the mutual adversarial network to calculate the loss value of the voiceprint recognition network. The output unit 405 may be configured to obtain a voiceprint recognition model based on the loss value and the voiceprint recognition network.

[0100] In this embodiment, the specific processing and technical effects of the sample acquisition unit 401, the network acquisition unit 402, the feature acquisition unit 403, the calculation unit 404, and the output unit 405 in the voiceprint recognition model training device 400 can be referred to respectively. Figure 1 The relevant descriptions of step 101, step 102, step 103, step 104, and step 105 in the corresponding embodiments are not repeated here.

[0101] In some optional implementations of this embodiment, the calculation unit 404 includes: a basic calculation module (not shown in the figure), a mutual confrontation calculation module (not shown in the figure), and a network calculation module (not shown in the figure). The basic calculation module can be configured to input the speaker feature and the non-speaker feature into the basic network to calculate the loss value of the basic network. The mutual confrontation calculation module can be configured to input the speaker feature and the non-speaker feature into the mutual confrontation network to calculate the loss value of the mutual confrontation network. The network calculation module can be configured to calculate the loss value of the voiceprint recognition network based on the loss value of the basic network and the loss value of the mutual confrontation network.

[0102] In some optional implementations of this embodiment, the basic network includes: an identity prediction module and a reconstruction module, the identity prediction module is used to perform speaker prediction on the speaker feature, and the reconstruction module is used to reconstruct the selected recognition vector feature of the non-speaker feature; the basic calculation module includes: an identity calculation submodule (not shown in the figure), a removal submodule (not shown in the figure), a series connection submodule (not shown in the figure), and a basic calculation submodule (not shown in the figure). Among them, the identity calculation submodule can be configured to input the speaker feature into the identity prediction module, obtain the speaker probability output by the identity prediction module, and calculate the identity loss value of the identity prediction module. The removal submodule can be configured to remove a portion of the speaker feature in the speaker feature to obtain the remaining feature. The series connection submodule can be configured to input the remaining feature and the non-speaker feature in series into the reconstruction module, obtain the reconstructed feature output by the reconstruction module, and calculate the reconstruction loss value of the reconstruction module. The basic calculation submodule can be configured to calculate the loss value of the basic network based on the identity loss value and the reconstruction loss value.

[0103] In some optional implementations of this embodiment, the mutual adversarial network includes: a non-speaker prediction module and a speaker prediction module, the non-speaker prediction module is used to perform non-speaker prediction on speaker features, the speaker prediction module is used to perform speaker prediction on non-speaker features, and the mutual adversarial calculation module includes: a first calculation submodule (not shown in the figure), a second calculation submodule (not shown in the figure), and a mutual adversarial calculation submodule (not shown in the figure). Among them, the first calculation submodule can be configured to input the speaker feature into the non-speaker prediction module, obtain the non-speaker feature output by the non-speaker prediction module, and calculate the loss value of the non-speaker prediction module. The second calculation submodule can be configured to input the non-speaker feature into the speaker prediction module, obtain the speaker feature output by the speaker prediction module, and calculate the loss value of the speaker prediction module. The mutual adversarial calculation submodule can be configured to calculate the loss value of the mutual adversarial network based on the loss value of the non-speaker prediction module and the loss value of the speaker prediction module.

[0104] In some optional implementations of the present embodiment, the sample acquisition unit 401 includes: an acquisition module (not shown in the figure), a framing module (not shown in the figure), a front-end extraction module (not shown in the figure), and a high-end extraction module (not shown in the figure). Among them, the acquisition module can be configured to acquire at least one audio data. The framing module can be configured to frame the audio data for each audio data in the at least one audio data to obtain multiple audio frames. The front-end extraction module can be configured to extract acoustic features from each audio frame in the multiple audio frames to obtain a feature sequence, wherein the feature sequence is used to characterize the acoustic features of multiple audio frames of the audio data. The high-end extraction module can be configured to extract identification features from the feature sequence to obtain identification vector features of the audio data.

[0105] In some optional implementations of this embodiment, the sample acquisition unit 401 includes: an acquisition module (not shown in the figure), a labeling module (not shown in the figure), a framing module (not shown in the figure), a sequence extraction module (not shown in the figure), a training module (not shown in the figure), and an input module (not shown in the figure). The acquisition module can be configured to acquire multiple audio data. The labeling module can be configured to perform speaker labeling on each audio data in the multiple audio data to obtain speaker labeling information. The framing module can be configured to perform framing on the audio data to obtain multiple audio frames of the audio data. The sequence extraction module can be configured to perform acoustic feature extraction on each audio frame in the multiple audio frames to obtain a feature sequence corresponding to the audio data, wherein the feature sequence is used to characterize the acoustic features of multiple audio frames of the audio data. The training module can be configured to use the feature sequence corresponding to each audio data and the speaker labeling information of each audio data to train a pre-constructed feature extraction network to obtain a feature extraction model, and the feature extraction network is used to characterize the corresponding relationship between the feature sequence and the recognition vector feature. The above-mentioned input module can be configured to input the feature sequences corresponding to the respective audio data in the multiple audio data into the feature extraction model in sequence, so as to obtain a feature sample set including multiple recognition vector features.

[0106] The voiceprint recognition model training device provided by the embodiment of the present disclosure, first, the sample acquisition unit 401 acquires a feature sample set including at least one recognition vector feature; secondly, the network acquisition unit 402 acquires a pre-established voiceprint recognition network, and the voiceprint recognition network includes: a feature extractor, a basic network and a mutual adversarial network; thirdly, the feature acquisition unit 403 inputs the recognition vector feature selected from the feature sample set into the feature extractor to obtain the speaker feature and non-speaker feature corresponding to the selected recognition vector feature; thirdly, the calculation unit 404 inputs the speaker feature and the non-speaker feature into the basic network, and inputs the speaker feature and the non-speaker feature into the mutual adversarial network to calculate the loss value of the voiceprint recognition network; finally, the output unit 405 obtains the voiceprint recognition model based on the loss value and the voiceprint recognition network. Therefore, by constructing a voiceprint recognition network including a feature extractor, a basic network, and a mutual adversarial network, the feature extractor can be used to extract speaker features and non-speaker features, the basic network can be used to predict speaker features, and the mutual adversarial network can be used to distinguish speaker features and non-speaker features, so that speaker features and non-speaker features can be represented separately, and speaker features and non-speaker features can be differentiated as much as possible in the voiceprint recognition network training until the two are balanced, so as to achieve the purpose of purer speaker features, thereby improving the accuracy of voiceprint recognition by the voiceprint recognition model.

[0107] Further references Figure 5As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a voiceprint recognition device. Figure 3 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0108] like Figure 5 As shown, the voiceprint recognition device 500 provided in this embodiment includes: an acquisition unit 501 and a recognition unit 502. The acquisition unit 501 can be configured to acquire the recognition vector feature of the speech to be recognized. The recognition unit 502 can be configured to input the recognition vector feature of the speech to be recognized into the above Figure 3 In the voiceprint recognition model generated by the device described in the embodiment, the voiceprint recognition result of the speech to be recognized is output.

[0109] In this embodiment, in the voiceprint recognition device 500, the specific processing of the acquisition unit 501 and the conversion unit 502 and the technical effects thereof can be referred to in Figure 3 The relevant descriptions of step 301 and step 302 in the corresponding embodiment are not repeated here.

[0110] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0111] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0112] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0113] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0114] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a voiceprint recognition model training method or a voiceprint recognition method. For example, in some embodiments, the voiceprint recognition model training method or the voiceprint recognition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the voiceprint recognition model training method or the voiceprint recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the voiceprint recognition model training method or the voiceprint recognition method in any other appropriate manner (for example, by means of firmware).

[0116] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable voiceprint recognition model training device, a voiceprint recognition device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0118] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0121] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0123] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A voiceprint recognition model training method, the method comprising: Acquire a feature sample set including at least one identification vector feature; Acquire a pre-established voiceprint recognition network, the voiceprint recognition network comprising: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in input information, the basic network is used to predict the speaker features; the mutual adversarial network is used to distinguish the speaker features from the non-speaker features; Perform the following training steps: Input the recognition vector features selected from the feature sample set into the feature extractor to obtain speaker features and non-speaker features corresponding to the selected recognition vector features; input the speaker features and the non-speaker features into the basic network, and input the speaker features and the non-speaker features into the mutual adversarial network to calculate the loss value of the voiceprint recognition network; based on the loss value and the voiceprint recognition network, obtain a voiceprint recognition model.

2. According to the method of claim 1, the step of inputting the speaker feature and the non-speaker feature into the basic network and the mutual adversarial network respectively, and calculating the loss value of the voiceprint recognition network comprises: Inputting the speaker feature and the non-speaker feature into the basic network, and calculating the loss value of the basic network; Inputting the speaker feature and the non-speaker feature into the mutual adversarial network, and calculating a loss value of the mutual adversarial network; Based on the loss value of the basic network and the loss value of the mutual adversarial network, the loss value of the voiceprint recognition network is calculated.

3. The method according to claim 2, wherein: The basic network includes: an identity prediction module and a reconstruction module, wherein the identity prediction module is used to perform speaker prediction on the speaker feature, and the reconstruction module is used to reconstruct the selected recognition vector feature of the non-speaker feature; the speaker feature and the non-speaker feature are input into the basic network, and the loss value of the basic network is calculated, including: Inputting the speaker feature into the identity prediction module, obtaining the speaker probability output by the identity prediction module, and calculating the identity loss value of the identity prediction module; Removing a portion of the speaker's features to obtain the remaining features; Inputting the remaining features and the non-speaker features in series into the reconstruction module to obtain the reconstructed features output by the reconstruction module, and calculating the reconstruction loss value of the reconstruction module; Based on the identity loss value and the reconstruction loss value, a loss value of the base network is calculated.

4. The method according to claim 2, wherein: The mutual adversarial network includes: a non-speaker prediction module and a speaker prediction module, wherein the non-speaker prediction module is used to perform non-speaker prediction on speaker features, and the speaker prediction module is used to perform speaker prediction on non-speaker features, and the speaker features and the non-speaker features are input into the mutual adversarial network, and the loss value of the mutual adversarial network is calculated, including: Inputting the speaker feature into the non-speaker prediction module to obtain the non-speaker feature output by the non-speaker prediction module, and calculating the loss value of the non-speaker prediction module; Inputting the non-speaker feature into the speaker prediction module to obtain the speaker feature output by the speaker prediction module, and calculating the loss value of the speaker prediction module; The loss value of the mutual adversarial network is calculated based on the loss value of the non-speaker prediction module and the loss value of the speaker prediction module.

5. The method according to any one of claims 1 to 4, wherein: The acquiring of a feature sample set including at least one identification vector feature comprises: Collecting at least one audio data; For each audio data in the at least one audio data, the audio data is divided into frames to obtain a plurality of audio frames; Extracting acoustic features of each audio frame from the plurality of audio frames to obtain a feature sequence, wherein the feature sequence is used to characterize acoustic features of the plurality of audio frames of the audio data; The identification features are extracted from the feature sequence to obtain the identification vector features of the audio data.

6. The method according to any one of claims 1 to 4, wherein: The acquiring of a feature sample set including at least one identification vector feature comprises: Collect multiple audio data; For each audio data among the multiple audio data, speaker tagging is performed on the audio data to obtain speaker tagging information; Divide the audio data into frames to obtain multiple audio frames of the audio data; Extracting acoustic features of each audio frame in the multiple audio frames to obtain a feature sequence corresponding to the audio data, wherein the feature sequence is used to characterize acoustic features of multiple audio frames of the audio data; Using feature sequences corresponding to each audio data and speaker annotation information of each audio data, a pre-constructed feature extraction network is trained to obtain a feature extraction model, wherein the feature extraction network is used to characterize the correspondence between the feature sequence and the recognition vector feature; The feature sequences corresponding to the respective audio data in the plurality of audio data are sequentially input into the feature extraction model to obtain a feature sample set including a plurality of recognition vector features.

7. A voiceprint recognition method, the method comprising: Obtaining recognition vector features of the speech to be recognized; The recognition vector feature of the speech to be recognized is input into a voiceprint recognition model generated by the method according to any one of claims 1 to 6, and the voiceprint recognition result of the speech to be recognized is output.

8. A voiceprint recognition model training device, the device comprising: A sample acquisition unit, configured to acquire a feature sample set including at least one recognition vector feature; The network acquisition unit is configured to acquire a pre-established voiceprint recognition network, wherein the voiceprint recognition network includes: a feature extractor, a basic network and a mutual adversarial network; the feature extractor is used to extract speaker features and non-speaker features in input information, the basic network is used to predict the speaker features; the mutual adversarial network is used to distinguish the speaker features from the non-speaker features; A feature acquisition unit is configured to input the recognition vector feature selected from the feature sample set into the feature extractor to obtain speaker features and non-speaker features corresponding to the selected recognition vector feature; A calculation unit is configured to input the speaker feature and the non-speaker feature into the basic network, and input the speaker feature and the non-speaker feature into the mutual adversarial network, and calculate the loss value of the voiceprint recognition network; The output unit is configured to obtain a voiceprint recognition model based on the loss value and the voiceprint recognition network.

9. The device according to claim 8, wherein: The computing unit comprises: A basic calculation module is configured to input the speaker feature and the non-speaker feature into the basic network and calculate the loss value of the basic network; A mutual adversarial calculation module is configured to input the speaker feature and the non-speaker feature into the mutual adversarial network and calculate a loss value of the mutual adversarial network; The network calculation module is configured to calculate the loss value of the voiceprint recognition network based on the loss value of the basic network and the loss value of the mutual resistance network.

10. The device according to claim 9, wherein: The basic network includes: an identity prediction module and a reconstruction module, wherein the identity prediction module is used to perform speaker prediction on the speaker feature, and the reconstruction module is used to reconstruct the identification vector feature selected from the non-speaker feature; the basic calculation module includes: an identity calculation submodule, configured to input the speaker feature into the identity prediction module, obtain the speaker probability output by the identity prediction module, and calculate the identity loss value of the identity prediction module; A removal submodule is configured to remove a portion of the speaker's features to obtain remaining features; A concatenation submodule, configured to input the remaining features and the non-speaker features in series into the reconstruction module to obtain the reconstructed features output by the reconstruction module, and calculate the reconstruction loss value of the reconstruction module; The basic calculation submodule is configured to calculate the loss value of the basic network based on the identity loss value and the reconstruction loss value.

11. The device according to claim 9, wherein: The mutual adversarial network includes: a non-speaker prediction module and a speaker prediction module, wherein the non-speaker prediction module is used to perform non-speaker prediction on speaker features, and the speaker prediction module is used to perform speaker prediction on non-speaker features, and the mutual adversarial calculation module includes: a first calculation submodule, configured to input the speaker feature into the non-speaker prediction module, obtain the non-speaker feature output by the non-speaker prediction module, and calculate the loss value of the non-speaker prediction module; a second calculation submodule, configured to input the non-speaker feature into the speaker prediction module, obtain the speaker feature output by the speaker prediction module, and calculate the loss value of the speaker prediction module; The mutual adversarial calculation submodule is configured to calculate the loss value of the mutual adversarial network based on the loss value of the non-speaker prediction module and the loss value of the speaker prediction module.

12. The device according to any one of claims 8 to 11, wherein: The sample acquisition unit comprises: A collection module, configured to collect at least one audio data; A framing module is configured to frame each audio data in the at least one audio data to obtain a plurality of audio frames; A front-end extraction module is configured to extract acoustic features from each of the multiple audio frames to obtain a feature sequence, wherein the feature sequence is used to characterize the acoustic features of the multiple audio frames of the audio data; The high-end extraction module is configured to extract identification features from the feature sequence to obtain identification vector features of the audio data.

13. The device according to any one of claims 8 to 11, wherein: The sample acquisition unit comprises: A collection module is configured to collect a plurality of audio data; A tagging module is configured to tag the audio data with a speaker for each of the plurality of audio data to obtain speaker tagging information; A framing module is configured to frame the audio data to obtain multiple audio frames of the audio data; A sequence extraction module is configured to extract acoustic features from each of the multiple audio frames to obtain a feature sequence corresponding to the audio data, wherein the feature sequence is used to characterize acoustic features of multiple audio frames of the audio data; A training module is configured to train a pre-built feature extraction network using feature sequences corresponding to each audio data and speaker annotation information of each audio data to obtain a feature extraction model, wherein the feature extraction network is used to characterize the correspondence between the feature sequence and the recognition vector feature; The input module is configured to input the feature sequences corresponding to the respective audio data in the plurality of audio data into the feature extraction model in sequence, so as to obtain a feature sample set including a plurality of recognition vector features.

14. A voiceprint recognition device, comprising: An acquisition unit, configured to acquire a recognition vector feature of the speech to be recognized; The recognition unit is configured to input the recognition vector feature of the speech to be recognized into the voiceprint recognition model generated by the device according to any one of claims 8 to 13, and output the voiceprint recognition result of the speech to be recognized.

15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.

Images