Speaker recognition method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a speaker recognition method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-tested voice and a sample voice, and the sample voice comprises a registration voice, a training voice and a test voice; extracting a plurality of target speaker embedding vectors from the to-be-tested voice and the registered voice, and generating a target speaker embedding vector pair set; calculating a voice similarity score according to the target speaker embedded vector pair set; and recognizing a speaker corresponding to the voice to be detected according to the voice similarity score. According to the method, speaker recognition in a complex acoustic scene can be realized under the condition that a large number of computing resources are not consumed.

Description

Technical Field

[0001] The present application relates to the technical field of speech analysis, and in particular to a speaker recognition method and device, an electronic device and a storage medium. Background Art

[0002] Speaker recognition has produced extensive and important applications in identity authentication and security verification. Its purpose is to automatically identify the speaker's identity information from speech through computers. Speaker recognition can be divided into text-related and text-independent according to whether it relies on fixed speech content; it can be divided into speaker identification and speaker confirmation according to the application scenario. Speaker identification is a multiple-choice task. By comparing the test voiceprint features with the pre-registered voiceprint features, the identity of the speaker matching the test voice is obtained. Speaker confirmation is a one-to-one task to determine whether the speaker identity declared in the test voice is true. However, the current speaker recognition system is limited by the mismatch problem between registration and test scenarios in practical applications. In practical applications, facing complex acoustic scenarios will cause the system recognition performance to be significantly reduced.

[0003] In order to reduce the performance impact caused by mismatch, IDVC (Inter-dataset variability compensation) and score calibration are traditionally used as scene compensation methods, which achieve compensation by decomposing speaker and scene changes in the speaker vector space. However, this method does not fully utilize the statistical information of each scene, and the computational overhead is large, which is still not enough to solve the robustness and generalization problems of speaker recognition systems.

[0004] Therefore, how to better improve the robustness and generalization performance of the speaker recognition system in complex acoustic scenarios without consuming a lot of computing resources has become an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a speaker recognition method to solve the problem of insufficient robustness and generalization performance of a speaker recognition system in complex acoustic scenarios.

[0006] Correspondingly, the embodiments of the present application also provide a speaker recognition device, an electronic device and a storage medium to ensure the implementation and application of the above method.

[0007] In order to solve the above problems, the present application discloses a speaker recognition method, including:

[0008] Obtain the voice to be tested and the sample voice, where the sample voice includes the registration voice, the training voice and the test voice;

[0009] Extracting a number of target speaker embedding vectors from the test speech and the registration speech respectively, and generating a set of target speaker embedding vector pairs;

[0010] Compute a speech similarity score based on the set of target speaker embedding vector pairs;

[0011] Identify the speaker corresponding to the speech to be tested based on the speech similarity score.

[0012] Optionally, a speech similarity score is calculated based on the target speaker embedding vector pair, including:

[0013] Calculate the intra-pair distances of the target speaker embedding vector pairs in the target speaker embedding vector pair set to obtain an intra-pair distance set of the target speaker embedding vector pairs;

[0014] Convert the intra-pair distance set of the target speaker embedding vector pair into a normalized distance set;

[0015] Obtain a set of quality indicators of the voice to be tested and the registered voice;

[0016] The alignment score is calculated as the speech similarity score based on the normalized distance set and the quality indicator set.

[0017] Optionally, the intra-pair distance set of the target speaker embedding vector pair is converted to a normalized distance set, including:

[0018] Obtain training speech from sample speech;

[0019] Extracting a plurality of first speaker embedding vector pairs and second speaker embedding vector pairs from the training speech to generate a first embedding vector pair set and a second embedding vector pair set;

[0020] The first embedding vector pair set and the second embedding vector pair set are used to adaptively score normalize the intra-pair distance set of the target speaker embedding vector pairs to generate a normalized distance set of the target speaker embedding vector pairs.

[0021] Optionally, obtain the quality indicator set of the voice to be tested and the registered voice, including:

[0022] The quality indicators of the tested speech and the registered speech are calculated respectively to obtain a set of quality indicators of the tested speech and the registered speech; the quality indicators include effective audio length, total speech length and speech signal-to-noise ratio.

[0023] Optionally, the speaker recognition method further includes:

[0024] generating a third set of embedding vector pairs according to the test speech of the sample speech;

[0025] Calculating quality indicators of the third embedding vector pair set, including effective audio length, total speech length, and speech signal-to-noise ratio of the test speech corresponding to the third embedding vector pair set;

[0026] According to the third embedding vector pair set and the corresponding quality indicators, adaptive weights corresponding to the normalized distance set and the quality indicator set are determined.

[0027] Optionally, generating a third set of embedding vector pairs according to the test speech of the sample speech includes:

[0028] Obtaining a test voice of a sample voice;

[0029] Fitting statistical properties of test speech;

[0030] Extracting training speech of the sample speech according to the probability distribution of the statistical property of the fitted test speech;

[0031] A plurality of third embedding vector pairs are extracted from the extracted training speech to generate a set of third embedding vector pairs.

[0032] Optionally, identifying the speaker corresponding to the speech to be tested according to the speech similarity score includes:

[0033] Get the speech similarity score;

[0034] Determine the registration voice that is most similar to the voice to be tested according to the voice similarity score;

[0035] The speaker corresponding to the registered voice that is most similar to the voice to be tested is determined as the speaker corresponding to the voice to be tested.

[0036] The present application also discloses a speaker recognition device, which includes:

[0037] An acquisition module is used to acquire the speech to be tested and the sample speech, where the sample speech includes the registration speech, the training speech and the test speech;

[0038] An extraction module is used to extract a number of target speaker embedding vectors from the test speech and the registration speech respectively, and generate a set of target speaker embedding vector pairs;

[0039] A calculation module, for calculating a speech similarity score based on a set of target speaker embedding vector pairs;

[0040] The recognition module is used to identify the speaker corresponding to the speech to be tested according to the speech similarity score.

[0041] The embodiment of the present application also discloses an electronic device, including: a processor; and a memory, on which executable codes are stored. When the executable codes are executed, the processor executes one or more speaker recognition methods in the embodiments of the present application.

[0042] The embodiments of the present application also disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, a processor executes one or more speaker recognition methods in the embodiments of the present application.

[0043] Compared with the prior art, the embodiments of the present application have the following advantages:

[0044] In an embodiment of the present application, a speech to be tested and a sample speech are obtained, and the sample speech includes a registered speech, a training speech, and a test speech. A number of target speaker embedding vectors are respectively extracted from the speech to be tested and the registered speech, and a set of target speaker embedding vector pairs is generated to capture the characteristics of the speaker in the speech to be tested, in preparation for subsequent speaker recognition. The speech similarity score is calculated based on the set of target speaker embedding vector pairs, and the speaker corresponding to the speech to be tested is identified based on the speech similarity score. In an embodiment of the present application, it is possible to score the speech to be tested based on the similarity between the speech to be tested and the registered speech without consuming a large amount of computing resources, and confirm the speaker corresponding to the speech to be tested based on the score, thereby realizing speaker recognition in complex acoustic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of a speaker recognition method embodiment of the present application;

[0046] Figure 2 is a schematic diagram of an embodiment of a speaker recognition method of the present application;

[0047] Figure 3 is a schematic diagram of another embodiment of a speaker recognition method of the present application;

[0048] Figure 4 is a schematic diagram of another embodiment of a speaker recognition method of the present application;

[0049] Figure 5 It is a structural block diagram of an embodiment of a speaker recognition device of the present application. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0051] Reference Figure 1 , is a flowchart of a speaker recognition method embodiment of the present application, comprising the following steps:

[0052] Step 101: Acquire a speech to be tested and a sample speech, wherein the sample speech includes a registration speech, a training speech, and a test speech;

[0053] The speech to be tested may be a speech signal input into a speaker recognition system and required for speaker recognition. Among the sample speech, the registration speech may be a speech sample recorded and submitted by the speaker when registering, which can be used for system recognition and adaptation to the specific speech features of the speaker. In the speaker recognition system, it can be used for speaker identity authentication. The training speech can be obtained through an external data set and used to enable the speaker recognition system to learn and adapt to different speech features and patterns during the training phase, thereby improving the recognition accuracy. The test speech can be collected by the specific business side when verifying the algorithm or obtained through feedback after applying the algorithm, and is used to evaluate the performance and accuracy of the system. It can detect the performance of the system when processing different speech inputs and make corresponding adjustments and optimizations.

[0054] Step 102: extracting a plurality of target speaker embedding vectors from the test speech and the registration speech respectively, and generating a set of target speaker embedding vector pairs;

[0055] The speaker embedding vector is a low-dimensional, fixed-size vector representation that converts the speech signal into a high-dimensional feature space. It can effectively capture the speaker's voice characteristics and play a key role in the subsequent speaker recognition task. For a test speech and a registered speech, they can be sent to the speaker representation extraction module, which extracts N speaker embedding vectors from the test speech and the registered speech respectively to obtain the test speech embedding vector e t ∈R D and the registered speech embedding vector e s ∈R D , R D represents a D-dimensional real number set, e t ∈R D Indicates e t is a D-dimensional real vector, e S ∈R D Indicates e s is a D-dimensional real number vector. Then, N target speaker embedding vector pairs are generated to form the target speaker embedding vector pair set {(e t,i ,e s,i )|i=1,…,N}, N is an integer greater than 1, corresponding to the number of backbone networks in the representation extraction module, and i represents the first target speaker embedding vector pair in the set. Common speaker embedding vector extraction methods include various speaker feature extractors such as d-vector, x-vector, s-vector, and xi-vector.

[0056] Step 103, calculating a speech similarity score according to the target speaker embedding vector pair set;

[0057] The speech similarity score may be a score calculated based on the similarity between the speech to be tested and the registered speech, and is used to determine the registered speech that is most similar to the speech to be tested.

[0058] Step 104: Identify the speaker corresponding to the speech to be tested according to the speech similarity score.

[0059] The embodiment of the present application is realized through the above steps, by obtaining the voice to be tested and the sample voice, the sample voice includes the registration voice, the training voice and the test voice. Figure 2 As shown, the acquired speech to be tested and the registered speech in the sample speech are formed into speech pairs, which are then input into the speaker representation extraction module, and several target speaker embedding vectors are extracted from the speech to be tested and the registered speech respectively, and a set of target speaker embedding vector pairs is generated to capture the speaker sound features corresponding to the speech to be tested. Where N is an integer greater than one. The speaker representation extraction module is composed of multiple backbone networks, and the number of target speaker embedding vector pairs extracted by the speaker representation extraction module is equal to the number of backbone networks in the speaker representation extraction module. The extracted N target speaker embedding vector pairs are input into the back-end scoring module, and the back-end scoring module calculates the speech similarity score based on the set of target speaker embedding vector pairs, measures the similarity between the embedding vector of the speech to be tested and the embedding vector corresponding to the reserved registered speech, and recalls the speaker identity corresponding to the registered speech with the highest similarity from the registration database based on the speech similarity score, and finally identifies the speaker corresponding to the speech to be tested, and outputs the speaker recognition result. Without consuming a lot of computing resources, it can score the test speech according to the similarity between the test speech and the registered speech, confirm the speaker corresponding to the test speech according to the score, and realize speaker recognition in complex acoustic scenarios.

[0060] Exemplary, reference Figure 3 , we can use the open source dataset VoxCeleb2 as the training set, use the online noise addition method, the noise addition probability can be set to 0.6, and train 7 different ResNet structures as the backbone network to form the speaker representation extraction module. The training process adopts a two-step training method. In the first step, we can use AAMSoftmax, the joint loss function of K-Subcenter and Inter-TopK, or the Sphereface2 loss function for training. Figure 3The √ mark in the figure indicates that the corresponding loss function is used for training. The same model can be divided into two groups, a and b, using different loss functions, such as 4a and 4b. The scale and margin of AAMSoftamx can be set to 32 and 0.2 respectively, the number of centers of K-Subcenter can be set to 3, the penalty of Inter-TopK and TopK can be set to 0.06 and 5 respectively, and the government weight of Sphereface2 can be set to 0.7 and 0.2 respectively; in the second step of training, only AAMSoftmax and K-subcenter are used as joint loss functions, the edge penalty of AAMSoftmax can be set from 0.2 to 0.5, and the speech duration of this step of training can be 6s. The first step of training of the 7 sub-networks can be trained for a total of 150 rounds, and the parameters of the last 10 rounds are averaged. The second step of training can be trained for 5 rounds.

[0061] Optionally, in an embodiment of the present application, the step of calculating the speech similarity score according to the target speaker embedding vector pair set includes:

[0062] Calculating the intra-pair distances of the target speaker embedding vector pairs in the target speaker embedding vector pair set to obtain the intra-pair distance set of the target speaker embedding vector pairs;

[0063] The intra-pair distance of the target speaker embedding vector pair can represent the similarity between the two embedding vectors of the test speech and the registration speech in the target speaker embedding vector pair. There are many methods for calculating the intra-pair distance of the target speaker embedding vector pair, including calculating cosine similarity, Euclidean distance, log-likelihood ratio, etc., which are not limited in this application.

[0064] Converting the intra-pair distance set of the target speaker embedding vector pair into a normalized distance set;

[0065] Acquire a quality indicator set of the voice to be tested and the registered voice;

[0066] Among them, the quality of speech data directly affects the accuracy and reliability of speaker recognition. High-quality speech data can provide clear speech features and help improve the performance of the recognition system. On the contrary, low-quality speech data may cause recognition errors and affect the accuracy of the system. Speech quality is affected by many factors, and the corresponding quality indicators used to evaluate speech quality can be effective audio length, total speech length, signal-to-noise ratio, noise, etc.

[0067] A calibration score is calculated according to the normalized distance set and the quality indicator set of the speech to be tested and the registered speech as the speech similarity score.

[0068] Among them, the calibration score is the score obtained by regularizing the score distribution of the normalized distance set of the target speaker embedding vector pair according to the obtained quality indicator set, and the score obtained after calibration conforms to the log-likelihood ratio.

[0069] In this embodiment, the intra-pair distance of the target speaker embedding vector pair in the target speaker embedding vector pair set is calculated. For example, the log-likelihood ratio can be used as the similarity index for calculation to obtain the speech embedding vector e t With the registered speech embedding vector e s The log-likelihood ratio llr.

[0070]

[0071] Where p(e t ,e s |H 0 ) represents the probability that the tested speech and the registered speech belong to different speakers, p(e t ,e s |H 1 ) The probability that the test speech and the registered speech belong to the same speaker. The intra-pair distance set S of the target speaker embedding vector pair is obtained, S = {s i = <e t,i ,e s,i >|i=1,…,N}. i The intra-pair distance of the i-th target speaker embedding vector pair in the intra-pair distance set S of the target speaker embedding vector pair. It means that the intra-pair distance set of the target speaker embedding vector pair is normalized and converted into the normalized distance set S AS Obtaining the quality indicator set of the test speech and the registered speech, calculating the calibration score according to the normalized distance set and the quality indicator set, and using the calibration score as the final speech similarity score can better improve the robustness of the system for speaker recognition. For example, refer to Figure 2 , N speaker embedding vector pairs can be input into the backend scoring module to obtain the intra-pair distance set of the target speaker embedding vector pair, and then adaptive score normalization (ASNorm) is performed on the intra-pair distance set of the target speaker embedding vector pair. Adaptive score normalization is a method for improving score normalization technology, which can reduce the instability that may occur during the normalization process and improve the stability of speech recognition tasks. In addition, the adaptive mechanism enables the model to better generalize to different data distributions and improve the generalization ability of the model.

[0072] Optionally, in an embodiment of the present application, converting the intra-pair distance set of the target speaker embedding vector pair into a normalized distance set includes:

[0073] Acquire training speech from the sample speech;

[0074] Among them, the speech data in the open source dataset VoxCeleb2 can be obtained as the training speech.

[0075] Extracting a plurality of first speaker embedding vector pairs and second speaker embedding vector pairs from the training speech to generate a first embedding vector pair set and a second embedding vector pair set;

[0076] The training speech may be randomly sampled, and a number of speaker embedding vector pairs may be extracted from the sampled training speech to generate a first embedding vector pair set and a second embedding vector pair set.

[0077] The first embedding vector pair set and the second embedding vector pair set are used to adaptively score normalize the intra-pair distance set of the target speaker embedding vector pairs to generate a normalized distance set of the target speaker embedding vector pairs.

[0078] In this embodiment, a training speech is obtained from the sample speech, the training speech is randomly extracted, the speaker embedding vector is extracted from the extracted training speech, and a first embedding vector pair set Ψ is constructed. 1 and the second set of embedding vector pairs Ψ 2 , using Ψ 1 and 2 The ASNorm is used to preliminarily regularize the intra-pair distance set of the target speaker embedding vector pair to obtain the normalized distance set S of the target speaker embedding vector pair. AS .

[0079]

[0080] in, is a universal quantification symbol, used to mean "for all" or "for any one", 1,i Denotes the first embedding vector pair set Ψ 1 The i-th vector in , μ represents the mean calculation, and σ represents the standard deviation calculation.

[0081] By introducing an adaptive mechanism, adaptive score normalization can reduce the instability that may occur during the normalization process and improve the stability of the training process. In addition, the adaptive mechanism enables the model to better generalize to different data distributions, improving the generalization ability of the speaker recognition model.

[0082] Optionally, in an embodiment of the present application, the step of obtaining a quality indicator set of the voice to be tested and the registered voice includes:

[0083] The quality indicators of the speech to be tested and the registered speech are calculated respectively to obtain a set of quality indicators of the speech to be tested and the registered speech; the quality indicators include effective audio length, total speech length and speech signal-to-noise ratio.

[0084] In this embodiment, the quality of the test speech and the registered speech is affected by multiple factors, corresponding to multiple quality indicators. The corresponding effective audio length, total speech length and speech signal-to-noise ratio can be calculated for each speech pair of the test speech and the registered speech as QMFs indicators. QMFs (Quality Measure functions) is a function used to evaluate the performance of data, models or systems. It can be used in different fields and tasks to measure and describe the quality or performance of the evaluated object in a quantitative manner. Among them, the effective audio length refers to the part of the speech that actually contains useful speech information. The effective audio length will directly affect the performance of the speaker recognition system. If the effective speech length is short, it may lead to insufficient feature extraction, thereby affecting the final speaker recognition accuracy. The total speech length includes effective speech and invalid speech. Invalid speech includes silence, meaningless background noise, etc. If the total speech length is too long, it will increase the computational burden of speaker recognition, and may also bring more background noise or other interference factors, thereby affecting the robustness of the final speaker recognition. The speech signal-to-noise ratio (SNR) refers to the ratio of speech signal to background noise. The higher the speech signal-to-noise ratio, the clearer the speech signal, the less background noise, and the better the speech quality. By calculating the effective audio length, total speech length, and speech signal-to-noise ratio of the test speech and the registered speech, we can obtain the quality indicator set of the test speech and the registered speech, which can be used as scene information for subsequent score calibration and regularization of the speech similarity scores of the test speech and the registered speech.

[0085] Optionally, in an embodiment of the present application, the method further includes:

[0086] Generating a third set of embedding vector pairs according to the test speech in the sample speech;

[0087] Calculating quality indicators of the third embedding vector pair set, including effective audio length, total speech length, and speech signal-to-noise ratio of the test speech corresponding to the third embedding vector pair set;

[0088] According to the third embedding vector pair set and the corresponding quality indicator, adaptive weights corresponding to the normalized distance set and the quality indicator set are determined.

[0089] In this embodiment, a test speech in the sample speech can be obtained, and a small amount of known test speech can be used to construct a third speaker embedding vector pair to generate a third embedding vector pair set Ψ 3Calculating the quality index of the third embedding vector pair set includes calculating the effective audio length of the test speech corresponding to the third embedding vector pair set, calculating the total length of the test speech, and calculating the signal-to-noise ratio of the test speech. In the training process of the speaker recognition model, the third embedding vector pair set Ψ can be used 3 To train the adaptive weights of the model. For example, the third embedding vector pair set Ψ can be calculated 3 The internal distance set of 1 and 2 For the third embedding vector pair set Ψ 3 The ASNorm is used to preliminarily regularize the internal distance set to obtain S′, and then the LLR (Log-Likelihood Ratio) loss function C is used. llr For the regularized third embedding vector pair set Ψ 3 The internal distance set is trained to obtain the adaptive weight v of the speaker recognition model 0 , W T and b.

[0090]

[0091] s″=v 0 s′+W T Q+b

[0092] where s i ∈S′,s j ∈S′,N tar With N non Represents each training batch, Ψ 3 The total number of speaker agreements and disagreements in , corresponding to the similarity score s in each batch i With s j s″ is the calibration score, s′∈S′, Q is the quality indicator set of the third embedding vector pair set, and the adaptive weight v 0 ,W∈R 6×1 ,b∈R 1 The LLR loss function is based on the concept of log-likelihood ratio, which can evaluate the performance of the model by comparing the difference between the probability distribution predicted by the model and the probability distribution of the true label. Using the LLR loss function to train the adaptive weights of the speaker recognition model can handle the problem of class imbalance, because it not only focuses on the model's prediction probability for the correct category, but also considers the relative size of the model's prediction probability for all categories. In the process of training iterations, each time s″ is obtained, it will be substituted into C llr Formula calculation, the obtained C llr The training weights are updated by obtaining the gradient of the training weights. After the training converges, the final adaptive weight v can be obtained. 0 , WT and b, which are used to calculate the calibration score when performing speaker recognition of the speech to be tested. Through the above steps, the weight of the speaker recognition model is adaptively adjusted using the intra-distance metric of the third embedding vector set, which can improve the reuse of the model, enhance the robustness and generalization performance of the speaker recognition system, and enable it to face more complex acoustic scenes. Moreover, the adjustment of the adaptive weight only requires a small amount of test speech, which can improve the robustness and generalization performance of the speaker recognition system without consuming a large amount of computing resources.

[0093] Optionally, in an embodiment of the present application, generating a third set of embedding vector pairs according to the test speech of the sample speech includes:

[0094] Acquiring a test voice of the sample voice;

[0095] fitting statistical properties of the test speech;

[0096] Among them, the statistical properties can be statistical indicators that reflect the characteristics of the data set, such as the concentration trend, dispersion degree, and distribution form. By calculating these statistical properties, the concentration trend, dispersion degree, and distribution of the data can be understood. The corresponding statistical properties of the test speech can be the total duration of the test speech, the average duration of the test speech, the duration variance of the test speech, the average signal-to-noise ratio of the test speech, etc.

[0097] Extracting training speech of the sample speech according to the probability distribution of the fitted statistical properties of the test speech;

[0098] A plurality of third embedding vector pairs are extracted from the extracted training speech to generate a third embedding vector pair set.

[0099] In this embodiment, the acquired test speech is regarded as a sample, and the statistical properties of the test speech, such as the average value of speech duration, the variance of duration, the average signal-to-noise ratio, etc., can represent the overall statistical properties of the actual data when the business side performs speaker recognition to a certain extent. Then, the statistical properties of the acquired test speech are fitted by methods such as kernel density estimation and maximum likelihood estimation, and random sampling is performed from the training speech using the fitted probability distribution. Sampling methods such as inverse transform sampling and Markov chain Monte Carlo sampling can be used. Finally, the sampling results are tested, and chi-square tests and Kolmogorov-Smirnov tests can be used. If the test gap is too large, resampling is required. By fitting the statistical properties of the acquired test speech, the distribution characteristics of the speech data in the actual application process of the business side can be better described. Then, the training speech is sampled according to the fitted probability distribution, and a number of third embedding vector pairs are extracted from the extracted training speech, so that a third embedding vector pair set that is more in line with the actual data can be obtained.

[0100] Exemplary, reference Figure 4 After completing the training of the entire speaker representation extraction module, we can extract speech pairs with similar probability distribution from the training set according to the statistical characteristics of the 1,000 test environment speech pairs to form a development set of 30,000 pairs for training the adaptive weights and QMFs score calibration unit in the back-end scoring module. The adaptive weights are trained for 100 rounds and the score calibration unit for 500 rounds. The actual test set can include complex acoustic scenes and events from movies, news interviews, different age groups, multiple voices, etc., across channels and multiple languages. Finally, we get Figure 4 The results are shown. EER (Equal Error Rate) is a commonly used performance evaluation indicator in speaker verification systems. It refers to the error rate when the false rejection rate (FRR) and the false acceptance rate (FAR) are equal in the speaker verification system. minDCF (Minimum Detection Cost Function) is a function that takes into account the costs of false rejection and false acceptance as well as the prior probabilities of the real speaker and the imposter. The lower the EER and minDCF, the better. Figure 4 The obtained test results show that the system using the fractional calibration process proposed in this application achieves the best performance, surpassing the other single backbone network systems in the test.

[0101] Optionally, in an embodiment of the present application, the step of identifying a speaker corresponding to the speech to be tested according to the speech similarity score includes:

[0102] Obtaining the speech similarity score;

[0103] Determining, according to the voice similarity score, a registration voice that is most similar to the voice to be tested;

[0104] The speaker corresponding to the registered voice that is most similar to the voice to be tested is determined as the speaker corresponding to the voice to be tested.

[0105] In this embodiment, according to the normalized distance set generated between the speech to be tested and the registered speech, a similarity score s″ between the speech to be tested and the registered speech can be calculated.

[0106] s″=v 0 s AS +W T Q+b

[0107] Among them, s AS ∈S AS , v 0 , W Tand b are the adaptive weights of the speaker recognition model obtained according to the test voice training. Q is the quality index set of the voice to be tested and the registered voice. According to the obtained similarity score, the similarity between the registered voice in the registration database and the voice to be tested can be determined. The higher the similarity, the greater the possibility that the speakers corresponding to the registered voice and the voice to be tested are the same speaker. Therefore, the speaker corresponding to the registered voice that is most similar to the voice to be tested can be determined as the speaker corresponding to the voice to be tested. Exemplarily, after obtaining the normalized distance set, the normalized distance set and the quality index set of the voice to be tested and the registered voice can be fused with the adaptive weight obtained by pre-training, and then score calibration is performed to calculate the calibration score, and the final speaker recognition result is output according to the calibration score. Exemplarily, a score threshold can also be set for the similarity score. When there is one or more registered voices with a similarity score higher than the score threshold to the voice to be tested, the speaker corresponding to the registered voice with the highest score is selected as the speaker of the voice to be tested. When there is no similarity score higher than the score threshold, speaker recognition is performed again.

[0108] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0109] On the basis of the above embodiments, this embodiment further provides a speaker recognition device, which is applied to electronic devices such as terminal devices and servers.

[0110] Reference Figure 5 , shows a structural block diagram of an embodiment of a speaker recognition device of the present application, which may specifically include the following modules:

[0111] An acquisition module 501 is used to acquire a speech to be tested and a sample speech, wherein the sample speech includes a registration speech, a training speech and a test speech;

[0112] An extraction module 502 is used to extract a plurality of target speaker embedding vectors from the test speech and the registration speech respectively, and generate a set of target speaker embedding vector pairs;

[0113] A calculation module 503, configured to calculate a speech similarity score based on the target speaker embedding vector pair set;

[0114] The recognition module 504 is used to recognize the speaker corresponding to the speech to be tested according to the speech similarity score.

[0115] Optionally, the calculation module includes:

[0116] The target speaker embedding vector pair inner distance calculation submodule is used to calculate the inner distance of the target speaker embedding vector pair in the target speaker embedding vector pair set to obtain the inner distance set of the target speaker embedding vector pair;

[0117] A normalization submodule, configured to convert a set of intra-pair distances of the target speaker embedding vector pair into a set of normalized distances;

[0118] A quality indicator set acquisition submodule, used to acquire the quality indicator sets of the voice to be tested and the registered voice;

[0119] The calibration score calculation submodule is used to calculate the calibration score as the speech similarity score according to the normalized distance set and the quality indicator set of the speech to be tested and the registered speech.

[0120] Optionally, the normalization submodule includes:

[0121] A training speech acquisition unit, used to acquire the training speech from the sample speech;

[0122] A first embedding vector pair set and a second embedding vector pair set generating unit, configured to extract a plurality of first speaker embedding vector pairs and second speaker embedding vector pairs from the training speech to generate a first embedding vector pair set and a second embedding vector pair set;

[0123] An adaptive score normalization unit is used to use the first embedding vector pair set and the second embedding vector pair set to adaptively score normalize the intra-pair distance set of the target speaker embedding vector pair to generate a normalized distance set of the target speaker embedding vector pair.

[0124] Optionally, the quality indicator set acquisition submodule is further used to:

[0125] The quality indicators of the speech to be tested and the registered speech are calculated respectively to obtain a set of quality indicators of the speech to be tested and the registered speech; the quality indicators include effective audio length, total speech length and speech signal-to-noise ratio.

[0126] Optionally, the speaker recognition device further includes:

[0127] A third embedding vector pair set generating module, configured to generate a third embedding vector pair set according to a test speech in the sample speech;

[0128] A third embedding vector pair set quality index calculation module, used to calculate the quality index of the third embedding vector pair set, including the effective audio length, total speech length and speech signal-to-noise ratio of the test speech corresponding to the third embedding vector pair set;

[0129] The adaptive weight determination module is used to determine the adaptive weights corresponding to the normalized distance set and the quality indicator set according to the third embedding vector pair set and the corresponding quality indicator.

[0130] Optionally, the third embedding vector pair set generating module includes:

[0131] A test speech acquisition submodule, used to acquire the test speech of the sample speech;

[0132] A fitting submodule, used for fitting the statistical properties of the test speech;

[0133] An extraction submodule, used for extracting the training speech of the sample speech according to the probability distribution of the fitted statistical properties of the test speech;

[0134] The third embedding vector pair set generation submodule is used to extract a plurality of third embedding vector pairs from the extracted training speech to generate a third embedding vector pair set.

[0135] Optionally, the identification module is further used for:

[0136] Obtaining the speech similarity score;

[0137] Determining, according to the voice similarity score, a registration voice that is most similar to the voice to be tested;

[0138] The speaker corresponding to the registered voice that is most similar to the voice to be tested is determined as the speaker corresponding to the voice to be tested.

[0139] A speaker recognition device provided by an embodiment of the present application can obtain a speech to be tested and a sample speech, wherein the sample speech includes a registration speech, a training speech, and a test speech. A plurality of target speaker embedding vectors are respectively extracted from the speech to be tested and the registration speech to generate a set of target speaker embedding vector pairs to capture the speaker sound features corresponding to the speech to be tested. A speech similarity score is calculated based on the set of target speaker embedding vector pairs to measure the similarity between the embedding vector of the speech to be tested and the embedding vector corresponding to the reserved registration speech, and the identity of the speaker corresponding to the registration speech with the highest similarity is recalled from the registration database based on the speech similarity score, and finally the speaker corresponding to the speech to be tested is identified.

[0140] An embodiment of the present application also provides a speaker recognition device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, one or more speaker recognition methods in the above embodiments are implemented.

[0141] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0142] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0143] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0144] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0145] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable speaker recognition terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable speaker recognition terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable speaker recognition terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable speaker recognition terminal device, so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0149] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0150] The speaker recognition method and device, an electronic device and a storage medium provided by the present application are introduced in detail above. The principles and implementation methods of the present application are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A speaker recognition method, characterized in that: The method comprises: Acquire a speech to be tested and a sample speech, wherein the sample speech includes a registration speech, a training speech, and a test speech; Extracting a plurality of target speaker embedding vectors from the test speech and the registration speech respectively, and generating a set of target speaker embedding vector pairs; Calculating a speech similarity score based on the target speaker embedding vector pair set; The speaker corresponding to the speech to be tested is identified according to the speech similarity score.

2. The method according to claim 1, characterized in that The calculating the speech similarity score according to the target speaker embedding vector pair set includes: Calculating the intra-pair distances of the target speaker embedding vector pairs in the target speaker embedding vector pair set to obtain the intra-pair distance set of the target speaker embedding vector pairs; Converting the intra-pair distance set of the target speaker embedding vector pair into a normalized distance set; Acquire a quality indicator set of the voice to be tested and the registered voice; A calibration score is calculated according to the normalized distance set and the quality indicator set of the speech to be tested and the registered speech as the speech similarity score.

3. The method according to claim 2, characterized in that The step of converting the intra-pair distance set of the target speaker embedding vector pair into a normalized distance set includes: Acquire training speech from the sample speech; Extracting a plurality of first speaker embedding vector pairs and second speaker embedding vector pairs from the training speech to generate a first embedding vector pair set and a second embedding vector pair set; The first embedding vector pair set and the second embedding vector pair set are used to adaptively score normalize the intra-pair distance set of the target speaker embedding vector pairs to generate a normalized distance set of the target speaker embedding vector pairs.

4. The method according to claim 2, characterized in that: The obtaining of the quality indicator set of the voice to be tested and the registered voice comprises: The quality indicators of the speech to be tested and the registered speech are calculated respectively to obtain a set of quality indicators of the speech to be tested and the registered speech; the quality indicators include effective audio length, total speech length and speech signal-to-noise ratio.

5. The method according to claim 2, characterized in that: The method further comprises: Generating a third set of embedding vector pairs according to the test speech in the sample speech; Calculating quality indicators of the third embedding vector pair set, including effective audio length, total speech length, and speech signal-to-noise ratio of the test speech corresponding to the third embedding vector pair set; According to the third embedding vector pair set and the corresponding quality indicator, adaptive weights corresponding to the normalized distance set and the quality indicator set are determined.

6. The method according to claim 5, characterized in that The step of generating a third set of embedding vector pairs according to the test speech of the sample speech comprises: Acquiring a test voice of the sample voice; fitting statistical properties of the test speech; Extracting training speech of the sample speech according to the probability distribution of the fitted statistical properties of the test speech; A plurality of third embedding vector pairs are extracted from the extracted training speech to generate a third embedding vector pair set.

7. The method according to claim 1, characterized in that The step of identifying a speaker corresponding to the speech to be tested according to the speech similarity score includes: Obtaining the speech similarity score; Determining, according to the voice similarity score, a registration voice that is most similar to the voice to be tested; The speaker corresponding to the registered voice that is most similar to the voice to be tested is determined as the speaker corresponding to the voice to be tested.

8. A speaker recognition device, characterized in that: include: An acquisition module is used to acquire the speech to be tested and the sample speech, wherein the sample speech includes the registration speech, the training speech and the test speech; An extraction module, used to extract a plurality of target speaker embedding vectors from the test speech and the registration speech respectively, and generate a set of target speaker embedding vector pairs; A calculation module, configured to calculate a speech similarity score based on the target speaker embedding vector pair set; The recognition module is used to recognize the speaker corresponding to the speech to be tested according to the speech similarity score.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the speaker recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speaker recognition method according to any one of claims 1 to 7 is implemented.

Images