Speaker recognition method and apparatus, electronic device, and storage medium

Abstract

The application discloses a speaker recognition method and device, electronic equipment and a storage medium. The speaker recognition method comprises the following steps: extracting a target voiceprint representation of a speaker; performing channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel; and comparing the first migrated voiceprint representation with the reference voiceprint representation to obtain a recognition result of the speaker. Through the above method, the accuracy of the speaker recognition result can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical voice technical field, in particular to a sounder recognition method and device, electronic equipment and storage medium. BACKGROUND

[0002] Sounder recognition refers to a technology of detecting the identity of a sounder by using the sound of the sounder, for example, voiceprint recognition technology. Compared with other biometric technologies, sounder recognition uses a voice signal to confirm the identity, has the characteristics of low cost, easy collection, easy storage, difficult imitation, friendly interaction, and can be remotely operated through a telephone or a network. Therefore, sounder recognition has good and broad application prospects in many fields such as home, security, military, and economy.

[0003] Sounder recognition can be divided into same-channel scenarios and cross-channel scenarios according to application scenarios. The voice of the sounder during registration is a reference voice, and the voice of the sounder during testing or application is a target voice. Same-channel sounder recognition refers to that the target voice and the reference voice are recorded by the same device, and the target voice and the reference voice belong to the same channel. Cross-channel sounder recognition refers to that the target voice and the reference voice are recorded by different devices, that is, the recording devices during registration and testing are changed, and the target voice and the reference voice belong to different channels. Since a new variable (channel) is introduced in cross-channel sounder recognition, the sounder recognition effect is affected.

[0004] At present, sounder recognition in a cross-channel scenario is often encountered in real application scenarios. Therefore, it is particularly important to solve the problem of sounder recognition in a cross-channel scenario and improve the sounder recognition effect. SUMMARY

[0005] The technical problem solved by the present application is to provide a sounder recognition method, device, electronic equipment and storage medium, which can improve the accuracy of the sounder recognition result.

[0006] To solve the above technical problem, the first aspect of the present application provides a sounder recognition method, which comprises: extracting a target voiceprint representation of a sounder; performing channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel; and comparing the first migrated voiceprint representation with the reference voiceprint representation to obtain a recognition result of the sounder.

[0007] To solve the above technical problems, the second aspect of the present application provides a speaker identification device, which comprises: an extraction module configured to extract a target voiceprint representation of a speaker; a channel migration module configured to perform channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel; and a comparison module configured to compare the first migrated voiceprint representation with the reference voiceprint representation to obtain an identification result of the speaker.

[0008] To solve the above technical problems, the third aspect of the present application provides an electronic device, which comprises a memory and a processor coupled to each other, the memory being configured to store program data, and the processor being configured to execute the program data to implement the aforementioned method.

[0009] To solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium, which stores program data, and the program data, when executed by a processor, is used to implement the aforementioned method.

[0010] The present application has the following beneficial effects: Different from the prior art, the present application extracts a target voiceprint representation of a speaker, and then performs channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, but the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel after channel migration. Thus, the first migrated voiceprint representation and the reference voiceprint representation can be compared in the same channel to obtain an identification result of the speaker, the problem of speaker identification in a cross-channel scenario is solved, and the accuracy of the speaker identification result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0012] Figure 1 is a flowchart of an embodiment of the speaker identification method of the present application;

[0013] Figure 2 is a flowchart of an embodiment of the speaker identification method of the present application; Figure 1

[0014] Figure 3 is a flowchart of an embodiment of the speaker identification method of the present application;

[0015] Figure 4 is a flowchart of an embodiment of the speaker identification method of the present application; Figure 3 ​is a flowchart of an embodiment of step S24 in the method for speaker recognition;

[0016] Figure 5 is a flowchart of an embodiment of the method for speaker recognition;

[0017] Figure 6 is Figure 5 is a flowchart of an embodiment of step S31 in the method for speaker recognition;

[0018] Figure 7 is a flowchart of pre-training of the voiceprint model;

[0019] Figure 8 is a flowchart of mixed training of two kinds of channel data;

[0020] Figure 9 is a flowchart of the cycle generative adversarial network in the present application;

[0021] Figure 10 is Figure 5 is a flowchart of an embodiment of step S32 in the method for speaker recognition;

[0022] Figure 11 is a flowchart of joint training of the voiceprint model and the channel migration model;

[0023] Figure 12 is Figure 5 is a flowchart of an embodiment of step S33 in the method for speaker recognition;

[0024] Figure 13 is a structural schematic block diagram of an embodiment of the speaker recognition device;

[0025] Figure 14 is a structural schematic block diagram of an embodiment of the electronic device;

[0026] Figure 15 is a structural schematic block diagram of an embodiment of the computer readable storage medium. DETAILED DESCRIPTION

[0027] Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to a single alternative embodiment. It is expressly understood that any of the embodiments described herein can be combined with any of the other embodiments.

[0028] The terms "first", "second", etc. are used only for descriptive purposes and are not to be construed as indicating or implying relative importance or an ordered ranking of the indicated technical features. Thus, features defined with "first", "second" can include at least one of the features explicitly or implicitly. In the description of the present application, the meaning of "a plurality of" is at least two, for example two, three, etc., unless otherwise explicitly and specifically limited. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a list of steps or units is not limited to the listed steps or units, but can optionally further include other steps or units not listed, or can optionally further include other steps or units inherent to such processes, methods, products, or apparatuses.

[0029] Recording device variation (across channels) is a difficult problem often encountered in speaker recognition field and is one of the current challenges. At present, the common practice is to obtain input of different channels to make the model more suitable for the test scene. For example, before a neural network, when a factor analysis technology is used, after a voiceprint representation is obtained, a PLDA (Probabilistic Linear Discriminant Analysis) model trained by cross-channel data is used to separate the channel space and the speaker space, thereby effectively dealing with the recording device variation scene. With deep neural networks gradually replacing traditional solutions, directly distinguishing cross-channel data is the most effective solution. However, a well-trained deep neural network model requires a large amount of training data, and in most cases, cross-channel data is lacking or unavailable. Regardless of which method uses cross-channel data, the same speaker must have data of two different channels at the same time.

[0030] Based on this, data augmentation is a common means when training data is lacking. For example, noise and reverberation are added, and speech synthesis technology is used to obtain more diverse training data, so that the model has better generalization ability. However, the commonly used noise and reverberation methods have limitations in channel removal. Noise and reverberation are still quite different from the channel, and it is difficult to achieve cross-channel data generation from the perspective of speech synthesis.

[0031] In view of the above, the present application provides a speaker recognition method. By performing channel migration on a target voiceprint representation, a first migrated voiceprint representation with the same channel as a reference voiceprint representation can be obtained. Thus, the first migrated voiceprint representation with the same channel can be compared with the reference voiceprint representation to obtain a speaker recognition result. In this way, the cross-channel problem is solved, and the speaker recognition result can be improved. In addition, in the training process, the same speaker no longer needs to have data of two different channels at the same time, thereby solving the problem of lack of training data in the cross-channel scene.

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0033] Please see Figures 1 to 2 , Figure 1 This is a flowchart illustrating an embodiment of the speaker identification method of this application. Figure 2 yes Figure 1 A flowchart illustrating an embodiment of step S11. The method may include the following steps:

[0034] Step S11: Extract the target voiceprint representation of the speaker.

[0035] The speaker can be a person, animal (such as a cat or dog), or any entity capable of producing sound. Optionally, the speaker may or may not be a living being.

[0036] In this embodiment, step S11 may include sub-steps S111 and S112:

[0037] Step S111: Obtain the target acoustic features of the speaker.

[0038] In one example, the process first involves acquiring the target speech data of the target speaker, then segmenting the target speech data into frames to obtain the corresponding speech frame sequence, and then pre-emphasizing the segmented speech frame sequence. Finally, the acoustic features of each speech frame are extracted sequentially to obtain the target acoustic features. These acoustic features refer to the feature data used to characterize the voiceprint information of the corresponding speech frame.

[0039] Acoustic features may include, but are not limited to, any one of the following: Fast Fourier Transform (FFT) features, pitch features, Linear Predictive Cepstral Coding (LPC) features, Mel-scale Frequency Cepstral Coefficients (MFCC) features, or Log Mel-filterbank (FBANK) features.

[0040] Step S112: Extract the voiceprint from the acoustic features of the target to obtain the voiceprint representation of the target.

[0041] In an example, the target acoustic feature can be input into the trained voiceprint model, and the voiceprint model can perform the step of extracting the target acoustic feature to obtain a voiceprint representation.

[0042] Step S12: Channel migration is performed on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel.

[0043] In an example, the target voiceprint representation can be input into a trained channel migration model, and the channel migration model can migrate the target voiceprint representation of the first channel to the second channel to obtain the first migrated voiceprint representation. The channel migration model can be a Cycle Gan, and in addition, the channel migration model can also be other derivative networks of the Cycle Gan or other migration networks, which are not limited here.

[0044] The reference voiceprint representation corresponds to reference speech data (or registered speech data). The reference speech data is a reference for verifying the identity of a speaker. If the target voiceprint representation is consistent with the reference voiceprint representation, it means that the speaker corresponding to the target voiceprint representation is the same as the speaker corresponding to the reference voiceprint representation, otherwise, they are not the same. Thus, the identity of the target speaker can be verified.

[0045] Step S13: The first migrated voiceprint representation is compared with the reference voiceprint representation to obtain a recognition result of the speaker.

[0046] In an example, the first migrated voiceprint representation and the reference voiceprint representation can be subjected to similarity calculation, and then based on the score, the recognition result of the speaker can be obtained. For example, when the score is greater than a preset score threshold, it means that the similarity between the first migrated voiceprint representation of the target speaker and the reference voiceprint representation is very high, and thus the recognition result of the speaker is the identity of the speaker corresponding to the reference voiceprint representation. The preset score threshold can be in the range of 80% to 100%, but is not limited thereto, and can be selected according to actual conditions.

[0047] In the above, the target voiceprint representation of the speaker is extracted, and then channel migration is performed on the target voiceprint representation to obtain the first migrated voiceprint representation, wherein the target voiceprint representation belongs to the first channel, but the first migrated voiceprint representation and the reference voiceprint representation belong to the second channel after channel migration. Thus, the first migrated voiceprint representation and the reference voiceprint representation can be compared in the same channel to obtain the recognition result of the speaker, which solves the problem of speaker recognition in a cross-channel scenario and can improve the accuracy of the recognition result of the speaker.

[0048] Please refer toFigures 3 to 4 , Figure 3 is a flowchart of an embodiment of a speaker identification method of the present application, Figure 4 is Figure 3 is a flowchart of an embodiment of step S24 in

[0049] Step S21: extracting a target voiceprint representation of a speaker.

[0050] For the description of this step, please refer to step S11 described above, which will not be repeated here.

[0051] Step S22: performing channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel.

[0052] For the description of this step, please refer to step S12 described above, which will not be repeated here.

[0053] Step S23: performing channel migration on the reference voiceprint representation to obtain a second migrated voiceprint representation, wherein the second migrated voiceprint representation belongs to the first channel.

[0054] Unlike the above embodiment, in this embodiment, the reference voiceprint representation can also be subjected to channel migration to migrate the reference voiceprint representation to the same channel as the target voiceprint representation, so as to realize same-channel comparison with the target voiceprint representation.

[0055] In an example, the reference voiceprint representation can be input into the trained channel migration model, and the channel migration model can migrate the reference voiceprint representation of the second channel to the first channel to obtain the second migrated voiceprint representation. In other embodiments, only the second migrated voiceprint representation and the target voiceprint representation can be compared to obtain the identification result of the speaker, so that same-channel comparison can also be realized.

[0056] Step S24: comparing the first migrated voiceprint representation and the reference voiceprint representation, and comparing the second migrated voiceprint representation and the target voiceprint representation to obtain the identification result of the speaker.

[0057] Specifically, the second migrated voiceprint representation and the target voiceprint representation are compared to obtain a first comparison result, and the first migrated voiceprint representation and the reference voiceprint representation are compared to obtain a second comparison result, and finally the first comparison result and the second comparison result are combined to obtain the identification result of the speaker.

[0058] In some embodiments, step S24 can include sub-steps S241-S243:

[0059] Step S241: performing similarity calculation on the second migrated voiceprint representation and the target voiceprint representation to obtain a first channel score.

[0060] Step S242: Similarity calculation is performed between the first migrated voiceprint representation and the reference voiceprint representation to obtain a second channel score.

[0061] In an example, the similarity calculation is, for example, a cosine similarity calculation, that is, a cosine value of two vectors is calculated as the first channel score and / or the second channel score. Alternatively, a value after normalization processing of the cosine value can also be taken as the first channel score and / or the second channel score.

[0062] Step S243: Based on the first channel score and the second channel score, an identification result of the speaker is obtained.

[0063] In an example, the first channel score and the second channel score can be calculated as an average value, for example, an arithmetic mean, a weighted average, etc.

[0064] Specifically, the fusion score can be calculated by using formula (1) as follows:

[0065]

[0066] wherein score is the fusion score, x is the target acoustic feature, y is the reference acoustic feature, T(x) is the target voiceprint representation, T(y) is the reference voiceprint representation, F(T(x)) is the first migrated voiceprint representation, G(T(y)) is the second migrated voiceprint representation, cos(T(x), G(T(y)) is the first channel score, and cos(F(T(x)), T(y)) is the second channel score.

[0067] Different from the above embodiment, in the present embodiment, the reference voiceprint representation is migrated to the first channel, and the target voiceprint representation is migrated to the second channel, respectively, then the first channel score is calculated for the target voiceprint representation, and the second channel score is calculated for the reference voiceprint representation, respectively, and then the first channel score and the second channel score are fused to obtain a more stable identification result.

[0068] In some embodiments, the speaker identification method is performed by multiple models, wherein the target voiceprint representation of the speaker is extracted by a voiceprint model, and the channel migration is performed by a channel migration model. The method for training the voiceprint model and the channel migration model will be introduced below.

[0069] Please refer to Figure 5 , Figure 5 is a flowchart of an embodiment of the speaker identification method of the present application. The method can include the following steps:

[0070] Step S31: The voiceprint model is pre-trained.

[0071] Step S32: obtaining a first sample voiceprint representation of the first sample acoustic feature by using the pre-trained voiceprint model, and training the channel migration model by using the first sample voiceprint representation.

[0072] Specifically, after pre-training the voiceprint model, the network parameters of the voiceprint model can be fixed first, only the channel migration model is trained, and when the channel migration model is trained stably, the network parameters of the voiceprint model are unfrozen for joint training.

[0073] Step S33: jointly training the voiceprint model and the channel migration model by using the second sample acoustic feature.

[0074] The first sample acoustic feature and the second sample acoustic feature each include sample acoustic features of the first channel and the second channel.

[0075] Please refer to Figures 6 to 8 , Figure 6 is Figure 5 a flowchart of an embodiment of step S31 in Figure 7 is a flowchart of pre-training of the voiceprint model of the present application, Figure 8 is a schematic diagram of mixed training of two kinds of channel data.

[0076] In the present embodiment, steps S311-S315 are an embodiment of the above-mentioned step S31.

[0077] Step S311: obtaining a plurality of third sample acoustic features of at least two channels, wherein the speakers corresponding to the plurality of third sample acoustic features are at least partially different.

[0078] In the present embodiment, the training data for the voiceprint model and the channel migration model contains data of two channels, but one speaker can not have data of two channels at the same time, in other words, the data of two channels can come from different speakers. Since it is no longer limited that the same speaker contains data of two channels, the problem of missing cross-channel training data can be solved.

[0079] The manner of obtaining the third sample acoustic feature is similar to the manner of obtaining the target acoustic feature in the above-mentioned step S111, which will not be described here.

[0080] Step S312: performing voiceprint extraction on each third sample acoustic feature by using the voiceprint model to obtain a third sample voiceprint representation of each third sample acoustic feature.

[0081] Step S313: predicting a speaker category probability of each third sample voiceprint representation.

[0082] As Figure 7As shown, in some implementations, the voiceprint model can be pre-trained using a Time-Delay Neural Network (TDNN) framework. During training, the TDNN may be followed by a statistical pooling layer and two fully connected layers connected in sequence. The first fully connected layer is used to obtain the voiceprint representation of the third sample, and the second fully connected layer, combined with the first fully connected layer, can be used to predict the speaker category probability of the third sample voiceprint representation. In practical applications, only the corresponding target voiceprint representation needs to be obtained; therefore, the voiceprint model includes a TDNN, a statistical pooling layer, and a fully connected layer connected in sequence.

[0083] TDNN is a frame-level network structure consisting of five fully connected layers with input and output node counts of 64-512-1536-512-512-1500 respectively. Each fully connected layer is followed by a ReLU activation layer and a Batch Normalization (BN) layer. Because xvector employs a frame-expanding strategy for the fully connected layer inputs, the number of expanded frames per layer is 5, 3, 3, 1, 1, and the number of skipped frames is 1, 2, 3, 1, 1 respectively. The output of TDNN is a frame-level speaker representation. Therefore, a statistical pooling layer can be connected after TDNN to transform the frame-level information into segment-level information. This is followed by two fully connected layers with input and output node counts of 3000-512-N, where N is the number of speaker classifications. Each fully connected layer is followed by a ReLU activation layer and a BN layer.

[0084] like Figure 7 As shown, the first fully connected layer is used to output voiceprint representations, such as reference voiceprint representation, target voiceprint representation, and third voiceprint representation.

[0085] In some implementations, the speaker category probability can specifically be the posterior probability of the speaker category.

[0086] like Figure 8 As shown, the training data includes two types of channel data, which correspond to the acoustic features (X acoustic features) of the first channel and the acoustic features (Y acoustic features) of the second channel, respectively. The pre-training process of the voiceprint model involves simultaneously inputting the X acoustic features and Y acoustic features into the voiceprint model (e.g., TDNN) for mixed training. Specifically, after passing through the first fully connected layer, the corresponding voiceprint representation 11 is obtained, and after passing through the second fully connected layer, the speaker category 12 of the first channel and the speaker category 13 of the second channel are obtained, respectively.

[0087] Step S314: Obtain the third loss based on the speaker category probability of the third sample voiceprint representation.

[0088] Specifically, the speaker category probabilities of the first channel and the second channel corresponding to the third sample voiceprint representation can be calculated separately, and then statistics can be performed to obtain the third loss.

[0089] Specifically, the third loss can be calculated using formula (2), as follows:

[0090] Loss class3 =-log(P T (X))-log(P T (Y)), formula (2)

[0091] Where X represents the acoustic characteristics of the first channel, Y represents the acoustic characteristics of the second channel, and P... T The probability of the speaker category corresponding to voiceprint model T is given below. class3 As the third loss, P T (X) represents the probability of the speaker category corresponding to the third acoustic feature of the first channel, P T (Y) represents the speaker category probability corresponding to the third acoustic feature of the second channel.

[0092] Step S315: Adjust the network parameters of the voiceprint model using the third loss.

[0093] Specifically, when the third loss is less than the preset loss threshold or the cumulative number of training iterations is greater than the preset number of iterations, the adjustment of the network parameters of the voiceprint model is stopped, and the pre-training of the voiceprint model is determined to be completed; otherwise, after adjusting the network parameters of the voiceprint model using the third loss, the voiceprint model is trained again until the initial training is completed.

[0094] Please see Figure 9 , Figure 9 This is a schematic diagram of the recurrent generative adversarial network (RGAN) in this application. Before introducing the joint training of the voiceprint model and the channel migration model, we will first briefly introduce the basic structure of the RGAN used in the channel migration model.

[0095] like Figure 9As shown, Cycle GAN is first applied to the style transfer of images, such as converting winter scenery into summer scenery, converting photos into oil painting style, etc. Cycle GAN is essentially two mirror-symmetrical GANs, which constitute a ring network. The two generators of the GANs are the G network and the F network, and the discriminators are the Dx and Dy networks, respectively. The generator is used for generating fake samples, and the discriminator is used for distinguishing true and false samples. The biggest feature of Cycle GAN is to use a ring consistency constraint strategy to associate the two GAN networks, so that the whole adversarial training effect is better.

[0096] Regarding loss calculation, adversarial loss (including the loss of discriminators and generators, respectively) needs to be calculated, and in addition, cycle consistency loss needs to be calculated to prevent the learned mappings G and F from contradicting each other.

[0097] The discriminator is used to effectively distinguish true and false samples, and the formula of the corresponding adversarial loss function is as follows:

[0098]

[0099] wherein, L GAN (F,D Y ,Y,X) is the loss function of the discriminator D y , the adversarial loss of the mapping function G: X->Y, x~pdata(x) is the data distribution of the X domain, and y~pdata(y) is the data distribution of the Y domain. Similarly, the adversarial loss L GAN (F,D x ,Y,X) of the mapping function F: Y->X can be obtained, that is, the loss function of the discriminator D X .

[0100] The formula for calculating the cycle consistency loss is as follows:

[0101]

[0102] wherein, L cyc (G,F) is the cycle consistency loss. For a real sample x, first, a fake sample is generated by using the first generator G. Then the fake sample is restored to the fake sample by using the second generator F. Finally, F(G(x))≈x is obtained through the equivalence constraint, and a similar strategy is used for the real sample y.

[0103] Finally, the total loss function of the whole Cycle Gan training can be represented by the following formula:

[0104]

[0105] wherein, L(G, F, D X , Y, X) is the loss function of the discriminator Dy, L Y (F, D GAN , Y, X) is the loss function of the discriminator Dx, L Y (G, F) is the cycle consistency loss, and λ is a weight value. The value of λ can be selected according to actual conditions, and is not limited herein. GAN x cyc

[0106] Please refer to Figures 10 to 11 , Figure 10 is Figure 5 a flowchart of an embodiment of step S32, Figure 11 is a flowchart of joint training of a voiceprint model and a channel migration model.

[0107] In this embodiment, steps S321-S325 are an embodiment of the above-described step S32.

[0108] Step S321: taking the voiceprint representation extracted by the voiceprint model from the first channel acoustic features as the first voiceprint representation, and taking the voiceprint representation extracted by the voiceprint model from the second channel acoustic features as the second voiceprint representation, the first channel acoustic features being the first sample acoustic features and the second sample acoustic features of the first channel, and the second channel acoustic features being the first sample acoustic features and the second sample acoustic features of the second channel.

[0109] As shown in Figure 11 , the first channel acoustic features (denoted as X acoustic features) are input into the voiceprint model to extract the first voiceprint representation (denoted as X xvector ) from the first channel acoustic features by using the voiceprint model, X xvector =T(X). The second channel acoustic features (denoted as Y acoustic features) are input into the voiceprint model to extract the second voiceprint representation (denoted as Y xvector ) from the second channel acoustic features by using the voiceprint model, Y xvector =T(Y).

[0110] Step S322: performing channel migration on the first voiceprint representation by using the first generation network of the channel migration model to obtain the third migration voiceprint representation of the second channel, and performing channel migration on the third migration voiceprint representation by using the second generation network of the channel migration model to obtain the fourth migration voiceprint representation of the first channel.

[0111] ​​​The channel migration model comprises two generation networks and two discrimination networks. The first generation network can migrate the voiceprint representation of the first channel to the voiceprint representation of the second channel, and conversely, the second generation network can migrate the voiceprint representation of the second channel to the voiceprint representation of the first channel.

[0112] In an embodiment, the first generation network F and the second generation network G can adopt the same structure, for example, a five-layer FN+ReLu structure with node numbers of 512-256-128-128-256-512, the first discriminator D X and the second discrimination network D Y may also adopt the same structure, for example, a three-layer FN+ReLu structure with node numbers of 512-128-64-2.

[0113] As shown in Figure 11 , the first voiceprint representation (X xvector ) of the first channel is input into the first generation network (denoted as F), and the third migrated voiceprint representation (denoted as ) of the second channel is output, then, is input into the second generation network (denoted as G), and the fourth migrated voiceprint representation (denoted as ) of the first channel is generated.

[0114] Step S323: performing channel migration on the second voiceprint representation by using the second generation network to obtain the fifth migrated voiceprint representation of the first channel, and performing channel migration on the fifth migrated voiceprint representation by using the first generation network to obtain the sixth migrated voiceprint representation.

[0115] Similarly, the second voiceprint representation (Y xvector ) of the second channel is input into the second generation network (denoted as G), and the fifth migrated voiceprint representation (denoted as ) of the first channel is output, then, is input into the first generation network (denoted as F), and the sixth migrated voiceprint representation (denoted as ) of the second channel is generated.

[0116] Step S324: obtaining the first loss by using the third migrated voiceprint representation, the fourth migrated voiceprint representation, the fifth migrated voiceprint representation and the sixth migrated voiceprint representation.

[0117] Among them, the first discriminator D X corresponding to the first generator and the second discriminator D Ya loss function of the first voiceprint representation and the second voiceprint representation, and a cycle consistency loss, and then synthesizes the three loss functions to obtain a total loss function of the channel migration model, so as to obtain the first loss.

[0118] Specifically, the first loss can be calculated by using the above formula (5), which will not be described here.

[0119] Step S325: Adjust the network parameters of the channel migration model by using the first loss.

[0120] Wherein, when the first loss is less than a preset loss threshold or the cumulative training number is greater than a preset number, the adjustment of the network parameters of the channel migration model is stopped, and it is determined that the preliminary training of the channel migration model is completed; otherwise, after the network parameters of the channel migration model are adjusted by using the first loss, the training of the channel migration model is continued until the preliminary training is completed. When the parameters are adjusted or updated, the same strategy as the Gan network is adopted, that is, the generation network and the discrimination network are alternately updated.

[0121] Please refer to Figure 12 , Figure 12 is Figure 5 the flowchart of an embodiment of step S33 in the method.

[0122] In this embodiment, steps S331-S334 are an embodiment of the above step S33.

[0123] Step S331: The speaker class probability corresponding to the first voiceprint representation, the speaker class probability corresponding to the second voiceprint representation, the speaker class probability corresponding to the third migration voiceprint representation, and the speaker class probability corresponding to the fifth migration voiceprint representation are predicted.

[0124] For how to predict the speaker class probability corresponding to the voiceprint representation, please refer to the above step S314, which will not be described here.

[0125] Step S332: Obtain the second loss based on the predicted speaker class probability.

[0126] Here, in addition to calculating the cross-entropy loss of the speaker class probability of the first voiceprint representation and the second voiceprint representation, the cross-entropy loss of the speaker class probability of the third migration voiceprint representation and the fifth migration voiceprint representation also needs to be calculated.

[0127] Specifically, the second loss can be calculated by using formula (6) as follows:

[0128] Loss class2 = -log(P T+F (X))-log(P T+G (Y))-log(P T (X))-log(PT (Y)) Formula (6)

[0129] wherein, Loss class2 is a second loss, P T+F is a probability of a speaker class corresponding to the voiceprint model T and the first generation network F, P T (X) is a probability of a speaker class corresponding to the first voiceprint representation, P T (Y) is a probability of a speaker class corresponding to the second voiceprint representation, P T+F (X) is a probability of a speaker class corresponding to the third transfer voiceprint representation corresponding to the voiceprint model T and the first generation network F, P T+G (Y) is a probability of a speaker class corresponding to the fifth transfer voiceprint representation corresponding to the voiceprint model T and the second generation network.

[0130] Step S333: adjusting the network parameters of the voiceprint model by using the second loss.

[0131] wherein, when the second loss is less than a preset loss threshold or the cumulative training number is greater than a preset number, the adjustment of the network parameters of the voiceprint model is stopped, and it is determined that the voiceprint model training is completed; otherwise, after the network parameters of the voiceprint model are adjusted by using the second loss, the voiceprint model continues to be trained until the training is completed.

[0132] In the embodiment, after the channel transfer model reaches stability, the voiceprint model is unfrozen, and the voiceprint model is trained together with the entire Gan network. When the parameters are updated, the voiceprint model is updated together with the generation network, the discrimination network is updated alone, and the two still adopt the alternating updating mode. In the joint training, the channel transfer model can still use the loss function of formula (5) to calculate the loss, and the network parameters of the channel transfer model are adjusted by using the obtained loss value.

[0133] In the above, the application proposes a framework combining a voiceprint model and Cycle Gan. When the recording equipment changes, it can also achieve good results under the condition of lacking cross-channel data, and saves the training cost.

[0134] Further, the voiceprint representation level is first adapted based on Cycle Gan. Compared with the data level, the network is easier to train and has better effect. Compared with the idea of Cycle Gan in the original data level, the method is easier to train and the training target is more direct. At the same time, the application first adopts the idea of joint learning of voiceprint model and Cycle Gan, thereby further improving the model effect.

[0135] Further, a voiceprint model and a more stable training strategy of Cycle Gan are proposed. In order to obtain better results, two-step training is adopted: first, pre-training the voiceprint model; second, preloading the voiceprint model to fix the parameters and training the Cycle Gan network; third, joint training and optimization of the whole network.

[0136] Further, a multi-model score fusion strategy is proposed. Specifically, the voiceprint representation is migrated to the first channel or the second channel, then the cosine scores of the target sample and the reference sample are calculated respectively, and finally the average method is adopted to obtain more stable recognition results.

[0137] Please refer to Figure 13 , Figure 13 is a structural schematic block diagram of an embodiment of the speaker identification device.

[0138] The speaker identification device 100 includes an extraction module 110, a channel migration module 120, and a comparison module 130. The extraction module 110 is configured to extract a target voiceprint representation of a speaker. The channel migration module 120 is configured to perform channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and a reference voiceprint representation belong to a second channel. The comparison module 130 is configured to compare the first migrated voiceprint representation with the reference voiceprint representation to obtain an identification result of the speaker.

[0139] In some embodiments, the channel migration module 120 is further configured to perform channel migration on the reference voiceprint representation to obtain a second migrated voiceprint representation, wherein the second migrated voiceprint representation belongs to the first channel; correspondingly, the comparison module 130 is further configured to compare the first migrated voiceprint representation with the reference voiceprint representation, and compare the second migrated voiceprint representation with the target voiceprint representation, to obtain the identification result of the speaker.

[0140] In some embodiments, the comparison module 130 is further configured to perform similarity calculation on the second migrated voiceprint representation and the target voiceprint representation to obtain a first channel score; perform similarity calculation on the first migrated voiceprint representation and the reference voiceprint representation to obtain a second channel score; and obtain the identification result of the speaker based on the first channel score and the second channel score.

[0141] In some embodiments, the extraction module 110 is further configured to obtain a target acoustic feature of the speaker; and perform voiceprint extraction on the target acoustic feature to obtain the target voiceprint representation.

[0142] In some embodiments, the target voiceprint representation of the speaker is extracted by a voiceprint model, and the channel migration is performed by a channel migration model.

[0143] In some embodiments, the speaker recognition method further comprises the following steps for training the voiceprint model and the channel migration model: pre-training the voiceprint model; obtaining a first sample voiceprint representation of the first sample acoustic feature by using the pre-trained voiceprint model, and training the channel migration model by using the first sample voiceprint representation; and jointly training the voiceprint model and the channel migration model by using the second sample acoustic feature.

[0144] In some embodiments, the channel migration model is a recurrent generative adversarial network, and / or the first sample acoustic feature and the second sample acoustic feature each comprises sample acoustic features of a first channel and a second channel, and the training of the channel migration model comprises: taking a voiceprint representation extracted by the voiceprint model from a first channel acoustic feature as a first voiceprint representation, and taking a voiceprint representation extracted by the voiceprint model from a second channel acoustic feature as a second voiceprint representation, the first channel acoustic feature being the first sample acoustic feature and the second sample acoustic feature of the first channel, and the second channel acoustic feature being the first sample acoustic feature and the second sample acoustic feature of the second channel; performing channel migration on the first voiceprint representation by using a first generative network of the channel migration model to obtain a third migration voiceprint representation of the second channel, and performing channel migration on the third migration voiceprint representation by using a second generative network of the channel migration model to obtain a fourth migration voiceprint representation of the first channel; performing channel migration on the second voiceprint representation by using the second generative network to obtain a fifth migration voiceprint representation of the first channel, and performing channel migration on the fifth migration voiceprint representation by using the first generative network to obtain a sixth migration voiceprint representation; obtaining a first loss by using the third migration voiceprint representation, the fourth migration voiceprint representation, the fifth migration voiceprint representation, and the sixth migration voiceprint representation; and adjusting network parameters of the channel migration model by using the first loss.

[0145] In some embodiments, in the process of jointly training the voiceprint model and the channel migration model, the method further comprises: predicting a speaker class probability corresponding to the first voiceprint representation, a speaker class probability corresponding to the second voiceprint representation, a speaker class probability corresponding to the third migration voiceprint representation, and a speaker class probability corresponding to the fifth migration voiceprint representation; obtaining a second loss based on the predicted speaker class probabilities; and adjusting network parameters of the voiceprint model by using the second loss.

[0146] In some embodiments, the pre-training of the voiceprint model comprises: obtaining a plurality of third sample acoustic features of at least two channels, wherein the speakers corresponding to the plurality of third sample acoustic features are at least partially different; extracting a third sample voiceprint representation of each third sample acoustic feature by using the voiceprint model, predicting a speaker class probability of each third sample voiceprint representation, obtaining a third loss based on the speaker class probabilities of the third sample voiceprint representations, and adjusting network parameters of the voiceprint model by using the third loss.

[0147] For the description of the above steps, please refer to the corresponding positions in the method embodiments, which will not be repeated here.

[0148] For the description of the above steps, please refer to the corresponding positions in the method embodiments, which will not be repeated here. Figure 14 Figure 14 is a structural schematic block diagram of an embodiment of the electronic device of the present application.

[0149] The electronic device 200 can include a memory 210 and a processor 220 coupled with each other, the memory 210 being configured to store program data, and the processor 220 being configured to execute the program data to implement the steps in any of the above method embodiments. The electronic device 200 can include, but is not limited to, a personal computer (e.g., a desktop computer, a notebook computer, a tablet computer, a palm computer, etc.), a mobile phone, a server, a wearable device, and an augmented reality (AR) device, a virtual reality (VR) device, a television, etc., without limitation.

[0150] Specifically, the processor 220 is configured to control itself and the memory 210 to implement the steps in any of the above method embodiments. The processor 220 can also be referred to as a CPU (Central Processing Unit). The processor 220 can be an integrated circuit chip having a processing capability of signals. The processor 220 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 220 can be implemented by multiple integrated circuit chips jointly.

[0151] For the description of the above steps, please refer to the corresponding positions in the method embodiments, which will not be repeated here. Figure 15 Figure 15 is a structural schematic block diagram of an embodiment of the computer-readable storage medium of the present application.

[0152] The computer-readable storage medium 300 stores program data 310, which is executed by a processor to implement the steps in any of the above method embodiments.

[0153] ​​The computer readable storage medium 300 can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which can store a computer program, or can be a server storing the computer program, which can send the stored computer program to other devices for running, or can run the stored computer program by itself.

[0154] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other manners. For example, the above-described device embodiments are merely illustrative, and the division of the modules or units can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0155] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0156] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0157] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0158] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for identifying a speaker, characterized in that, include: Extract the target voiceprint representation of the speaker; Channel migration is performed on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and the reference voiceprint representation belong to a second channel. The first migrated voiceprint representation is compared with the reference voiceprint representation to obtain the identification result of the speaker; The extraction of the target voiceprint representation of the speaker is performed by a voiceprint model, and the pre-training steps of the voiceprint model include: Acquire several third-sample acoustic features of at least two channels, wherein the speakers corresponding to the several third-sample acoustic features are at least partially different; The voiceprint model is used to extract the voiceprint of each of the third samples, thereby obtaining the voiceprint representation of each of the third samples. The probability of the speaker category is predicted for each of the third sample voiceprint representations; Based on the speaker category probability represented by the third sample voiceprint, a third loss is obtained; The network parameters of the voiceprint model are adjusted using the third loss.

2. The method according to claim 1, characterized in that, Before comparing the first migrated voiceprint representation with the reference voiceprint representation to obtain the speaker identification result, the method further includes: Channel migration is performed on the reference voiceprint representation to obtain a second migrated voiceprint representation, wherein the second migrated voiceprint representation belongs to the first channel; The comparison of the first migrated voiceprint representation with the reference voiceprint representation to obtain the speaker identification result includes: By comparing the first migrated voiceprint representation with the reference voiceprint representation, and the second migrated voiceprint representation with the target voiceprint representation, the identification result of the speaker is obtained.

3. The method according to claim 2, characterized in that, The comparison of the first migrated voiceprint representation with the reference voiceprint representation, and the comparison of the second migrated voiceprint representation with the target voiceprint representation, to obtain the speaker identification result includes: The similarity between the second migrating voiceprint representation and the target voiceprint representation is calculated to obtain the first channel score; The similarity between the first migrated voiceprint representation and the reference voiceprint representation is calculated to obtain the second channel score; The speaker's identification result is obtained based on the first channel score and the second channel score.

4. The method according to claim 1, characterized in that, The extraction of the target voiceprint representation of the speaker includes: Obtain the target acoustic features of the speaker; The acoustic features of the target are subjected to voiceprint extraction to obtain the voiceprint representation of the target.

5. The method according to claim 1, characterized in that, The extraction of the target voiceprint representation of the speaker is performed by the voiceprint model, and the channel migration is performed by the channel migration model.

6. The method according to claim 5, characterized in that, The method further includes the following steps to train the voiceprint model and the channel migration model: The voiceprint model is pre-trained; The first sample voiceprint representation of the acoustic features of the first sample is obtained by using the pre-trained voiceprint model, and the channel migration model is trained by using the first sample voiceprint representation. The acoustic features of the second sample are used to jointly train the voiceprint model and the channel migration model.

7. The method according to claim 6, characterized in that, The channel migration model is a cyclic generative adversarial network, and / or The first sample acoustic features and the second sample acoustic features both include sample acoustic features of the first channel and the second channel. Training the channel transfer model includes: The voiceprint representation obtained by the voiceprint model from the acoustic features of the first channel is used as the first voiceprint representation, and the voiceprint representation obtained by the voiceprint model from the acoustic features of the second channel is used as the second voiceprint representation. The acoustic features of the first channel are the first sample acoustic features and the second sample acoustic features of the first channel, and the acoustic features of the second channel are the first sample acoustic features and the second sample acoustic features of the second channel. The first voiceprint representation is channel-migrated using the first generation network of the channel migration model to obtain the third migrated voiceprint representation of the second channel, and the third migrated voiceprint representation is channel-migrated using the second generation network of the channel migration model to obtain the fourth migrated voiceprint representation of the first channel. The second voiceprint representation is channel-migrated using the second generation network to obtain the fifth migrated voiceprint representation of the first channel, and the fifth migrated voiceprint representation is channel-migrated using the first generation network to obtain the sixth migrated voiceprint representation. The first loss is obtained by using the third, fourth, fifth, and sixth migration voiceprint representations. The network parameters of the channel migration model are adjusted using the first loss.

8. The method according to claim 7, characterized in that, In the process of jointly training the voiceprint model and the channel migration model, the method further includes: The speaker category probability corresponding to the first voiceprint representation, the speaker category probability corresponding to the second voiceprint representation, the speaker category probability corresponding to the third migrated voiceprint representation, and the speaker category probability corresponding to the fifth migrated voiceprint representation are predicted. Based on the predicted speaker category probability, a second loss is obtained; The network parameters of the voiceprint model are adjusted using the second loss.

9. A speaker identification device, characterized in that, include: The extraction module is used to extract the target voiceprint representation of the speaker; A channel migration module is used to perform channel migration on the target voiceprint representation to obtain a first migrated voiceprint representation, wherein the target voiceprint representation belongs to a first channel, and the first migrated voiceprint representation and the reference voiceprint representation belong to a second channel. The comparison module is used to compare the first migrated voiceprint representation with the reference voiceprint representation to obtain the recognition result of the speaker; The extraction of the target voiceprint representation of the speaker is performed by a voiceprint model, and the pre-training steps of the voiceprint model include: Acquire several third-sample acoustic features of at least two channels, wherein the speakers corresponding to the several third-sample acoustic features are at least partially different; The voiceprint model is used to extract the voiceprint of each of the third samples, thereby obtaining the voiceprint representation of each of the third samples. The probability of the speaker category is predicted for each of the third sample voiceprint representations; Based on the speaker category probability represented by the third sample voiceprint, a third loss is obtained; The network parameters of the voiceprint model are adjusted using the third loss.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor coupled to each other, the memory being used to store program data and the processor being used to execute the program data to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images