Voiceprint recognition method and device, computer readable storage medium, and terminal

Abstract

A voiceprint recognition method and device, a computer readable storage medium and a terminal are provided. The method comprises the following steps: determining a first voiceprint feature vector of each of a plurality of registration voices belonging to the same registered person, and a posterior probability of each first voiceprint feature vector; determining a weight coefficient of each first voiceprint feature vector according to the posterior probability; performing a weighted operation on the first voiceprint feature vectors of the registration voices based on the determined weight coefficients to obtain an identity vector of the registered person; and performing voiceprint recognition on a voice to be recognized using the identity vectors of a plurality of registered persons. The accuracy and reliability of the determined identity vector of the registered person can be improved, thereby improving the accuracy of voiceprint recognition.

Description

Technical Field

[0001] This invention relates to the field of biometrics, and more particularly to a voiceprint recognition method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] Voiceprint recognition is a biometric technology that extracts speaker identity features from speech for authentication. It mainly consists of two stages: voiceprint registration and voiceprint authentication. The primary task of the registration stage is to extract the speaker's voiceprint feature vector from the recorded speech, primarily using traditional machine learning methods such as deep learning to represent speaker information. The main task of the authentication stage is to score the similarity between the voiceprint feature vector extracted from the test speech and the registrant's voiceprint feature vector for authentication. Currently, with the increasing demands for user experience in voiceprint recognition applications, registration and authentication speech is gradually shifting from long to short voice recordings, which places higher demands on voiceprint recognition technology in short voice environments. In short voice scenarios, registrants register their voiceprints by recording multiple short voice segments. How to improve the accuracy of voiceprint feature extraction and voiceprint recognition accuracy under limited registration voice resources is a pressing issue.

[0003] Against this backdrop, researchers have proposed various speaker recognition technologies for short speech environments. Among them, the short speech speaker recognition scheme based on the Identity Vector (Ivector) algorithm has relatively stable performance. It is not as demanding on the amount of training speech data and the quality of data labeling, and can adapt to various task requirements. Therefore, it is often used as a baseline standard to evaluate the performance of other technologies. Ivector is a speaker recognition method based on Gaussian Mixture Model-Universal Background Model (GMM-UBM). Specifically, the distribution of a person's voice features can be described by a Gaussian Mixture Model (GMM), and the distribution of voice features of many speakers can be described by a Universal Background Model (UBM). GMM-UBM adaptively obtains the speaker's GMM based on a certain segment of a person's speech using the maximum a posteriori probability estimation algorithm on the UBM, and is used to describe the speaker's voice feature distribution. Building upon this, Ivector models channel differences and speaker differences as a total factor matrix (Total Factor Matrix). It then combines the mean vectors of each order of the adaptively obtained Gaussian mean hypervector into a higher-dimensional mean hypervector. The projection of this Gaussian mean hypervector onto the total factor matrix is ​​used as the speaker's identity vector (i.e., Ivector). After obtaining the Ivectors from the registration and authentication phases, voiceprint authentication can be performed by calculating the similarity of the Ivectors.

[0004] However, the aforementioned implementation scheme, when performing voiceprint registration, chooses to perform mean pooling on the voiceprint features extracted from multiple speech segments of the same speaker (registrant), that is, using the result of mean pooling as the registrant's identity vector (specifically, this identity vector is used to represent the registrant's identity features). This mean pooling may lead to excessive smoothing of the Ivector features, especially in short speech environments. Since the registered speech is short, the reliability of the identity vector obtained based on the registered speech needs to be considered. For example, if there is one or more short speech segments with a very low signal-to-noise ratio (e.g., excessive environmental noise), the reliability of the voiceprint features extracted from such short speech segments will be low, which will affect the mean calculation result and thus reduce the accuracy of subsequent voiceprint authentication results. Summary of the Invention

[0005] The technical problem solved by this invention is how to improve the accuracy and reliability of the identified registrant's identity vector in short voice application scenarios, thereby improving the accuracy of voiceprint recognition.

[0006] To address the aforementioned technical problems, this invention provides a voiceprint recognition method, comprising the following steps: determining the first voiceprint feature vector of each of multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector; determining the weight coefficient of each first voiceprint feature vector based on the posterior probability; performing a weighted operation on the first voiceprint feature vectors of each registered voice based on the determined weight coefficient to obtain the identity vector of the registrant; and using the identity vectors of multiple registrants to perform voiceprint recognition on the voice to be recognized.

[0007] Optionally, determining the weight coefficient of each first voiceprint feature vector based on the posterior probability includes: comparing the posterior probability of each first voiceprint feature vector with a preset probability threshold; normalizing each posterior probability that is greater than or equal to the probability threshold, and using each normalized posterior probability as the weight coefficient of its respective first voiceprint feature vector.

[0008] Optionally, determining the weight coefficient of each first voiceprint feature vector based on the posterior probability further includes setting the weight coefficient of the first voiceprint feature vector to which each posterior probability belongs to is less than the probability threshold to 0.

[0009] Optionally, the step of using multiple registrant identity vectors to perform voiceprint recognition on the speech to be recognized includes: determining a second voiceprint feature vector of the speech to be recognized; determining the identity vector with the highest similarity to the second voiceprint feature vector; determining a similarity judgment threshold based on the scene category of the registrant's registered speech to which the identity vector with the highest similarity belongs and the scene category of the speech to be recognized; if the similarity between the identity vector with the highest similarity and the second voiceprint feature vector is greater than or equal to the similarity judgment threshold, then confirming that the speech to be recognized belongs to the registrant to which the identity vector with the highest similarity belongs; wherein, the combination of the scene category of the registered speech and the scene category of the speech to be recognized has a preset mapping relationship with the similarity judgment threshold.

[0010] Optionally, determining the identity vector with the highest similarity to the second voiceprint feature vector includes: using the probabilistic linear discriminant analysis (PLDA) algorithm to calculate the PLDA score between each registrant's identity vector and the second voiceprint feature vector; and taking the identity vector with the highest PLDA score between it and the second voiceprint feature vector as the identity vector with the highest similarity to the second voiceprint feature vector.

[0011] Optionally, before determining the identity vector with the highest similarity to the second voiceprint feature vector, the method further includes: performing dimensionality reduction processing on the identity vector and the second voiceprint feature vector of each registrant, and then performing length normalization.

[0012] Optionally, determining the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector, includes: for each registered voice, determining the posterior distribution of the voiceprint features of the registered voice in the full factor difference space, wherein the mean of the posterior distribution is used as the first voiceprint feature vector of the registered voice; determining the probability values ​​of the first voiceprint feature vectors of each of the other registered voices besides the first voiceprint feature vector of the registered voice on the posterior distribution; and taking the average of the determined probability values ​​as the posterior probability of the first voiceprint feature vector of the registered voice.

[0013] Optionally, for each registered speech, determining the posterior distribution of the speaker features of that registered speech in the full factor difference space includes: extracting acoustic features for each registered speech to determine the acoustic features of each registered speech; preprocessing the acoustic features to obtain a frame feature sequence for each registered speech; and determining the posterior distribution of the speaker features of that registered speech in the full factor difference space based on the frame feature sequence.

[0014] This invention also provides a voiceprint recognition device, comprising: a voiceprint feature vector determination module, configured to determine the first voiceprint feature vector of each of multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector; a weight coefficient determination module, configured to determine the weight coefficient of each first voiceprint feature vector based on the posterior probability; a registrant identity vector determination module, configured to perform a weighted operation on the first voiceprint feature vectors of each registered voice based on the determined weight coefficients to obtain the registrant's identity vector; and a voiceprint recognition module, configured to perform voiceprint recognition on the voice to be recognized using the identity vectors of multiple registrants.

[0015] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, executes the steps of the above-described voiceprint recognition method.

[0016] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the above-described voiceprint recognition method when running the computer program.

[0017] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0018] In this embodiment of the invention, the process of determining the registrant's identity vector takes into account the posterior probability of the first voiceprint feature vector of each registered speech of the registrant (the posterior probability can be used to indicate the credibility of the first voiceprint feature vector). Specifically: after determining the first voiceprint feature vector of multiple registered speech belonging to the same registrant, and the posterior probability of each first voiceprint feature vector, the weight coefficient of the first voiceprint feature vector of each registered speech is determined according to the posterior probability. Then, the first voiceprint feature vectors of multiple registered speech are weighted based on the weight coefficient, and finally fused to obtain the registrant's identity vector. Therefore, the accuracy and credibility of the determined registrant's identity vector can be effectively improved (i.e., the registrant's identity vector can more accurately represent the registrant's identity features), thereby improving the accuracy of the voiceprint recognition results.

[0019] Furthermore, in practical applications, for each registered speech, the posterior distribution of the determined voiceprint features in the full factorial difference space (e.g., this posterior distribution typically resembles a normal distribution curve) may not be entirely accurate. When the mean of the posterior distribution is used as the first voiceprint feature vector of the registered speech, the probability value of the first voiceprint feature vector on the posterior distribution (e.g., the probability value at the mean of the normal distribution) is also inaccurate. Therefore, in this embodiment of the invention, for the first voiceprint feature vector of each registered speech, the average calculation result of the probability values ​​of the first voiceprint feature vectors of other registered speech on the posterior distribution of the registered speech is used as the posterior probability of the first voiceprint feature vector of the registered speech. This effectively improves the accuracy of the determined posterior probability of each first voiceprint feature vector.

[0020] Furthermore, in this embodiment of the invention, an appropriate probability threshold can be preset, and the posterior probability of each first voiceprint feature vector can be compared with the probability threshold. It is understood that the first voiceprint feature vectors with posterior probabilities greater than or equal to the probability threshold have higher reliability, while those with posterior probabilities less than the probability threshold have lower reliability. Therefore, the posterior probabilities greater than or equal to the probability threshold can be normalized, and the normalized posterior probabilities can be used as the weight coefficients of their respective first voiceprint feature vectors. In addition, the weight coefficients of the first voiceprint feature vectors with posterior probabilities less than the probability threshold can be set to 0. Thus, the weight coefficients of each first voiceprint feature vector can be determined reasonably and accurately, improving the accuracy and reliability of the identity vector obtained through subsequent weighted fusion calculations.

[0021] Furthermore, the step of using multiple registrant identity vectors to perform voiceprint recognition on the speech to be recognized includes: determining a second voiceprint feature vector of the speech to be recognized; determining the identity vector with the highest similarity to the second voiceprint feature vector; determining a similarity decision threshold based on the scene category of the registrant's registered speech to which the identity vector with the highest similarity belongs and the scene category of the speech to be recognized; if the similarity between the identity vector with the highest similarity and the second voiceprint feature vector is greater than or equal to the similarity decision threshold, then confirming that the speech to be recognized belongs to the registrant to which the identity vector with the highest similarity belongs; wherein, there is a preset mapping relationship between the combination of the scene category of the registered speech and the scene category of the speech to be recognized and the similarity decision threshold.

[0022] In this embodiment of the invention, during the voiceprint authentication stage, the scene (environment) factors of the registered voice and the voice to be identified are considered. Specifically, this may include: first, determining the second voiceprint feature vector of the voice to be identified; then, determining the identity vector with the highest similarity to the second voiceprint feature vector; based on the combination of the scene category of the registered voice of the registrant to which the identity vector with the highest similarity belongs and the scene category of the voice to be identified (e.g., the registered voice belongs to the far field or near field, high signal-to-noise ratio or low signal-to-noise ratio, etc.), adaptively selecting the corresponding similarity decision threshold (e.g., the similarity decision threshold corresponding to a certain scene category combination can be determined by looking up a preset mapping table); and finally, determining the final voiceprint recognition result based on the comparison result between the highest similarity and the similarity decision threshold. This approach fully considers the influence of environmental factors on the similarity calculation result, which can further improve the accuracy of the voiceprint recognition result. Attached Figure Description

[0023] Figure 1 This is a flowchart of a voiceprint recognition method according to an embodiment of the present invention;

[0024] Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11;

[0025] Figure 3 yes Figure 1 A flowchart of a specific implementation of step S12;

[0026] Figure 4 yes Figure 1 A flowchart of a specific implementation of step S14;

[0027] Figure 5 This is a schematic diagram of the structure of a voiceprint recognition device according to an embodiment of the present invention. Detailed Implementation

[0028] As mentioned earlier, improving the accuracy of voiceprint feature extraction and voiceprint recognition is an urgent problem to be solved.

[0029] In existing technologies, short speech voiceprint recognition schemes based on the Ivector algorithm are often used as a baseline standard to evaluate the performance of other technical solutions. Ivector is based on a Gaussian Mixture-Universal Background Model (GMM-UBM). This model adaptively obtains the speaker's Gaussian Mixture Model (GMM) based on a segment of a person's speech using a maximum a posteriori probability estimation algorithm on the Universal Background Model (UBM), describing the speaker's voice feature distribution. Furthermore, it models channel differences and differences between speakers as a full-factor difference space, and combines the mean vectors of each order of the adaptively obtained GMM into a higher-dimensional mean hypervector. The projection of this Gaussian mean hypervector onto the full-factor space is used as the speaker's identity vector. After obtaining the identity vectors for the registration and authentication phases, voiceprint authentication can be performed by calculating the similarity of the identity vectors.

[0030] The inventors of this invention discovered through research that the above-mentioned implementation scheme, when performing voiceprint registration, selects to perform mean pooling on the voiceprint features extracted from multiple speech segments of the same speaker (i.e., the registrant), that is, using the result of mean pooling as the registrant's identity vector (specifically, this identity vector is used to represent the registrant's identity features). This mean pooling may lead to excessive smoothing of voiceprint features, especially in short speech environments. Since the registered speech is short, the reliability of the identity vector obtained based on the registered speech needs to be considered. For example, if there is a short speech segment or several short speech segments with a very low signal-to-noise ratio (e.g., excessive environmental noise), the reliability of the voiceprint features extracted from such short speech segments is very low, which will affect the mean calculation result, thereby reducing the accuracy of subsequent voiceprint authentication results.

[0031] To address the aforementioned technical problems, this invention provides a voiceprint recognition method, specifically comprising: determining the first voiceprint feature vector of each of multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector; determining the weight coefficient of each first voiceprint feature vector based on the posterior probability; performing a weighted operation on the first voiceprint feature vectors of each registered voice based on the determined weight coefficient to obtain the identity vector of the registrant; and using the identity vectors of multiple registrants to perform voiceprint recognition on the voice to be recognized.

[0032] Therefore, in determining the registrant's identity vector, this embodiment of the invention considers the posterior probability of the first voiceprint feature vector of each registered speech (the posterior probability can be used to indicate the credibility of the first voiceprint feature vector). Based on the posterior probability, a weight coefficient for the first voiceprint feature vector of each registered speech is determined. Then, based on the weight coefficient, a weighted operation is performed on the first voiceprint feature vectors of multiple registered speech samples, and finally, the registrant's identity vector is obtained by fusion. This effectively improves the accuracy and credibility of the determined registrant's identity vector, thereby improving the accuracy of the voiceprint recognition results.

[0033] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0034] Reference Figure 1 , Figure 1 This is a flowchart of a voiceprint recognition method according to an embodiment of the present invention. The method can be applied to various terminal devices with voiceprint recognition capabilities (for example, specifically including voiceprint registration and voiceprint authentication functions), such as computers, tablets, mobile phones, in-vehicle computing devices, smart wearable devices (such as smartwatches), as well as servers, cloud platforms, etc.

[0035] It is understood that, in specific implementations, the method can be implemented using a software program that runs in a processor integrated within the chip or chip module; or, the method can be implemented using hardware or a combination of hardware and software.

[0036] The voiceprint recognition method may include steps S11 to S14:

[0037] Step S11: Determine the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector;

[0038] Step S12: Determine the weight coefficient of each first voiceprint feature vector based on the posterior probability;

[0039] Step S13: Based on the determined weight coefficients, perform a weighted operation on the first voiceprint feature vector of each registered voice to obtain the identity vector of the registrant;

[0040] Step S14: Use the identity vectors of multiple registrants to perform voiceprint recognition on the speech to be recognized.

[0041] In the specific implementation of step S11, the multiple registration voice recordings (or multiple frames of registration voice recordings) of the same registrant can be multiple voice segments from the same speaker. The specific number of recordings / frames of the same registrant's voice recordings can be reasonably set according to the specific needs of the scenario.

[0042] The duration of each registered voice message can be the same or different, and the duration should be as moderate as possible, neither too long nor too short. It is understood that excessively long registered voice messages may reduce the efficiency of voiceprint feature extraction and increase computational overhead; conversely, excessively short voice messages may reduce the accuracy and reliability of the extracted voiceprint features due to insufficient voiceprint feature information. In specific implementations, the duration of each registered voice message can be reasonably limited according to the specific application scenario, and this embodiment of the invention does not impose such limitations.

[0043] The text content of each registered speech can also be set according to different application scenarios. For example, in text-related voiceprint recognition scenarios, the text content of multiple registered speech can be the same or similar (e.g., several sentences with a contextual relationship); in text-independent voiceprint recognition scenarios, the text content of multiple registered speech can be different.

[0044] In specific implementation, for the same registrant, the first voiceprint feature vector of each registered voice recording can be extracted after each recording, or the first voiceprint feature vector of each registered voice recording can be extracted separately after all multiple recordings have been completed. This embodiment of the invention does not impose any restrictions on this.

[0045] Reference Figure 2 , Figure 2 yes Figure 1 The flowchart shows a specific implementation of step S11 described above. Step S11 may include steps S21 to S23. Each step is described below.

[0046] In step S21, for each registered speech, the posterior distribution of the voiceprint features of the registered speech in the full factor difference space is determined, wherein the mean of the posterior distribution is used as the first voiceprint feature vector of the registered speech.

[0047] Furthermore, for each registered speech, determining the posterior distribution of the speaker features of that registered speech in the full factor difference space may include: extracting acoustic features for each registered speech to determine the acoustic features of each registered speech; preprocessing the acoustic features to obtain a frame feature sequence for each registered speech; and determining the posterior distribution of the speaker features of that registered speech in the full factor difference space based on the frame feature sequence.

[0048] The acoustic feature extraction can refer to adjusting the amplitude of each component in the spectrum of a sound segment as needed, based on the auditory perception mechanism, and parameterizing them to obtain a vector (i.e., acoustic features) suitable for representing the characteristics of the speech signal, using speech frames as units. The main steps of acoustic feature extraction can include pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), and filtering. The acoustic feature extraction algorithm can be selected from Mel-Frequency Cepstral Coefficients (MFCCs), FilterBank (FBank), and Perceptual Linear Prediction (PLP), or other existing algorithms capable of acoustic feature extraction. In specific implementations, the feature dimension can be set according to the needs of the specific application scenario; for example, in short speech recognition scenarios, the feature dimension can be set higher.

[0049] The preprocessing of the acoustic features may include, but is not limited to: cepstrum mean and variance normalization (CMVN), which can be used to eliminate signal-to-noise interference; differential processing, which can be used to obtain the correlation between speech frames; and voice activity detection (VAD), which can be used to remove silence. By preprocessing the extracted acoustic features, the dimensionality of the frame-level acoustic features can be increased, thereby improving the expressive power of the frame-level acoustic features.

[0050] In some non-limiting embodiments, after obtaining the frame feature sequence of each registered speech, a general Gaussian mixture-universal background model (GMM-UBM) and a T matrix can be used to determine the posterior distribution of the speaker features of each registered speech in the full factorial difference space. This can be implemented in a manner described in detail below.

[0051] Step 1: Adaptively obtain a Gaussian Mixture Model (GMM) for the speaker (i.e., the registrant) based on the GMM-UBM and Maximum A Posteriori (MAP) probability estimation algorithms. The speaker's voiceprint features are primarily contained in the GMM's mean vector. Therefore, the mean components of the GMM are concatenated into a high-dimensional mean hypervector. The first voiceprint feature vector for each registered speech is extracted based on this mean hypervector.

[0052] Among them, the Universal Background Model (UBM) is trained from background data, which comes from a large number of different speakers. The Gaussian Mixture Model (GMM) of a specific target speaker is obtained by adapting the UBM and the audio data of that speaker. In the above step one, the following formula (1) can be used to determine the mean supervector:

[0053] (1) M(i) = M(0) + T × w(i);

[0054] In the above formula, M(i) represents the mean supervector of the GMM calculated by the maximum a posteriori probability estimation algorithm for the i-th registered speech, M(i) ∈ R D×K , M(0) represents the mean supervector of the GMM-UBM, M(0) ∈ R D×K , T is a matrix used to represent the full factor difference space, T ∈ R D×K×F , w(i) represents the voiceprint feature (which can also be called the voiceprint vector) of the i-th registered speech with a prior distribution following N(0,1), R is the real number symbol, D is the dimension of the frame feature sequence, K is the mixture degree of the GMM, F is the dimension of the voiceprint feature of the i-th registered speech, F << D × K, and i is a positive integer.

[0055] Step two: Since T in formula (1) is not a square matrix, it is impossible to simply use matrix transformation to obtain the voiceprint feature / voiceprint vector of the registered speech. Instead, according to the GMM-UBM model, the posterior distribution of the voiceprint feature of the i-th registered speech in the full factor difference space is determined, and the mean of the posterior distribution is used as the first voiceprint feature vector of the i-th registered speech.

[0056] Specifically, given the i-th registered speech Y i , the mean supervector M(i) and variance R of the GMM calculated by the maximum a posteriori probability estimation algorithm k , the posterior distribution of the voiceprint feature of the i-th registered speech in the full factor difference space can be determined using the following formulas (2) to (4):

[0057]

[0058]

[0059] Solving formulas (2) and (3) can obtain formula (4):

[0060]

[0061] From the above, the posterior distribution of w(i) is a mean of l(i) -1 T T R-1 Γ y (i), with variance l(i). -1 The signal follows a Gaussian distribution; therefore, the first voiceprint feature vector of the i-th registered speech is the mean of the posterior distribution, i.e.:

[0062] Where, l(i) = T T Γ(i)R -1 T+I, Baumwech's zeroth-order statistic is The first-order statistic is

[0063] Wherein, P(w(i)∣Y i Let represent the posterior distribution of the voiceprint features / voiceprint vector of the i-th registered speech, and w(i) represent the voiceprint vector of the i-th registered speech. Let Y represent the estimated value of the first voiceprint feature vector of the i-th registered speech, i.e., the mean of the posterior distribution, and let argmax represent the function of the independent variable that maximizes the function value. i Let P(Y) represent the i-th registered voice. i |w(i)) represents the likelihood probability of the i-th registered speech given w(i), P(w(i)) represents the prior probability of w(i), ∝ represents the equivalence symbol, Π represents the cumulative multiplication symbol, ∑ represents the cumulative summation symbol, t represents the t-th frame of the feature sequence, and T i Let c represent the total number of frames in the feature sequence of the i-th registered speech, k identify the k-th GMM, K represent the mixing degree of the GMM, and c represent the number of frames in the feature sequence of the i-th registered speech. k This represents the weight of the k-th GMM. Indicates the symbol for the Gaussian function. M represents the feature sequence of the t-th frame of the i-th registered speech. k (i) represents the mean of the k-th Gaussian in the adaptive GMM, R k Let represent the variance of the GMM calculated by the maximum a posteriori probability estimation algorithm, and exp{} represent an exponential function with base e, l(i). -1 Let R represent the variance of the posterior distribution, l(i) represent the inverse of the variance of the posterior distribution, and Ri represent the variance of the posterior distribution. -1 Γ represents the inverse matrix of variance. y (i) represents a D×K dimensional statistic composed of K first-order statistics, and Γ(i) represents a statistic composed of K diagonal matrices γ. k (i)I D×D The components are concatenated to form a diagonal matrix, where I represents the D×D identity matrix, and γ... k (i) represents Baumwech's zeroth-order statistic. Ω represents the likelihood probability or state occupancy of the k-th GMM under a given speech sequence and the GMM-UBM model. 0) Γ represents the initial parameters of GMM-UBM. y,k (i) represents a first-order statistic, m k Let represent the mean of the k-th Gaussian in GMM-UBM.

[0064] After steps one and two, n registered voice records {Y} are determined. 1 ,Y 2 ,Y 3 ...Y n After the first voiceprint feature vector is obtained, the following step S22 is performed.

[0065] In step S22, the probability values ​​of the first voiceprint feature vectors of each of the other registered voices, excluding the first voiceprint feature vector of the registered voice, are determined on the posterior distribution.

[0066] In step S23, the average of the determined probability values ​​is used as the posterior probability of the first voiceprint feature vector of the registered speech.

[0067] In specific implementation, the method for determining the first voiceprint feature vector of each registered speech and the posterior probability of each first voiceprint feature vector can also be other appropriate existing methods, and the embodiments of the present invention do not limit this.

[0068] It is understandable that, in practical applications, for each registered speech, the posterior distribution of the determined voiceprint features in the full factorial difference space (e.g., this posterior distribution typically resembles a normal distribution curve) may not be entirely accurate. When the mean of the posterior distribution is used as the first voiceprint feature vector of the registered speech, the probability value of the first voiceprint feature vector on the posterior distribution (e.g., the probability value at the mean of the normal distribution) is also inaccurate. Therefore, in this embodiment of the invention, for the first voiceprint feature vector of each registered speech, the average calculation result of the probability values ​​of the first voiceprint feature vectors of other registered speech on the posterior distribution of the registered speech is used as the posterior probability of the first voiceprint feature vector of the registered speech. This effectively improves the accuracy of the determined posterior probability of each first voiceprint feature vector. (Continuing to refer to...) Figure 1 In the specific implementation of step S12, the weight coefficient of each first voiceprint feature vector is determined according to the posterior probability.

[0069] Reference Figure 3 , Figure 3 yes Figure 1A flowchart of a specific implementation of step S12 is provided. Step S12 may include steps S31 to S32, and each step is described below.

[0070] In step S31, the posterior probability of each first voiceprint feature vector is compared with a preset probability threshold.

[0071] The posterior probability of each first voiceprint feature vector can be used to indicate the credibility of that first voiceprint feature vector; specifically, it can indicate the degree to which the first voiceprint feature vector represents the identity characteristics of the registrant. The larger the posterior probability value, the higher the credibility of the first voiceprint feature vector to which that posterior probability belongs.

[0072] In practice, the probability threshold can be an empirical threshold obtained from the results of multiple historical voiceprint recognition experiments. The specific value of the probability threshold can be different in different application scenarios.

[0073] In step S32, each posterior probability that is greater than or equal to the probability threshold is normalized, and the normalized posterior probabilities are used as weight coefficients of the corresponding first voiceprint feature vector.

[0074] Furthermore, the weighting coefficients of the first voiceprint feature vectors to which each posterior probability is less than the probability threshold belongs can be set to 0. In specific implementations, they can also be set to other smaller values ​​greater than 0 and less than the probability threshold as weighting coefficients, so that the influence of the first voiceprint feature vectors to which each posterior probability is less than the probability threshold on the subsequent weighted fusion calculation results is minimized.

[0075] In this embodiment of the invention, by employing the method described above for determining weight coefficients based on posterior probabilities, an appropriate probability threshold can be preset. The posterior probability of each first voiceprint feature vector is then compared with the probability threshold. It is understood that first voiceprint feature vectors with posterior probabilities greater than or equal to the probability threshold have higher reliability, while those with posterior probabilities less than the probability threshold have lower reliability. Therefore, the posterior probabilities greater than or equal to the probability threshold can be normalized, and the normalized posterior probabilities can be used as the weight coefficients of their respective first voiceprint feature vectors. Furthermore, the weight coefficients of first voiceprint feature vectors with posterior probabilities less than the probability threshold can be set to smaller weight coefficients (e.g., 0). Thus, the weight coefficients of each first voiceprint feature vector can be determined reasonably and accurately, improving the accuracy and reliability of the identity vector obtained through subsequent weighted fusion calculations.

[0076] Continue to refer to Figure 1In the specific implementation of step S13, based on the determined weight coefficients, the first voiceprint feature vector of each registered voice is weighted to obtain the identity vector of the registrant.

[0077] In practice, the weight coefficients of each first voiceprint feature vector can be used to perform a weighted summation operation on each first voiceprint feature vector, and the vector obtained by the weighted summation can be used as the identity vector of the registrant.

[0078] In this embodiment of the invention, compared to the prior art which uses the result of mean pooling of the voiceprint feature vectors extracted from multiple voice segments of the same registrant as the registrant's identity vector, this mean pooling may lead to excessive smoothing of the Ivector features. Especially in short voice environments, due to the influence of environmental noise, the voiceprint feature vectors of one or more registered voice segments may have low reliability, thus affecting the mean calculation result and reducing the accuracy of subsequent voiceprint authentication results. In the process of determining the registrant's identity vector in this embodiment of the invention, the posterior probability of the first voiceprint feature vector of each registered voice segment of the registrant is considered. The weight coefficient of the first voiceprint feature vector of each registered voice segment is determined according to the posterior probability. Then, the first voiceprint feature vectors of multiple registered voice segments are weighted based on the weight coefficient, and finally fused to obtain the registrant's identity vector. Therefore, the accuracy and reliability of the determined identity vector can be effectively improved, that is, the registrant's identity vector can accurately represent the registrant's identity features.

[0079] In the specific implementation of step S14, the identity vectors of multiple registrants are used to perform voiceprint recognition on the speech to be recognized.

[0080] Reference Figure 4 , Figure 4 yes Figure 1 A flowchart illustrating a specific implementation of step S14. Step S14 may include steps S41 to S44, which are described below.

[0081] In step S41, the second voiceprint feature vector of the speech to be recognized is determined.

[0082] In specific implementation, the method for determining the second voiceprint feature vector of the speech to be recognized can be referred to Figure 1 The method for determining the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant in step S11 will not be described in detail here.

[0083] In step S42, the identity vector with the highest similarity to the second voiceprint feature vector is determined.

[0084] Further, step S42 may include: using the Probabilistic Linear Discriminant Analysis (PLDA) algorithm to calculate the PLDA score between each registrant's identity vector and the second voiceprint feature vector; and taking the identity vector with the highest PLDA score between it and the second voiceprint feature vector as the identity vector with the highest similarity to the second voiceprint feature vector.

[0085] The PLDA score can be used to indicate the similarity between the identity vector and the second voiceprint feature vector of the speech to be identified. The higher the PLDA score, the stronger the similarity; the lower the PLDA score, the weaker the similarity.

[0086] In specific implementations, other similarity scoring methods (e.g., cosine similarity scoring method) can be used to determine the identity vector with the highest similarity to the second voiceprint feature vector. This embodiment of the invention does not limit the similarity calculation method.

[0087] Furthermore, before step S42, the method described in this embodiment may also include: performing dimensionality reduction processing on the identity vector of each registrant and the second voiceprint feature vector, and then performing length normalization.

[0088] In practical implementation, a pre-trained Linear Discriminant Analysis (LDA) transformation matrix can be used for dimensionality reduction. The role of LDA is to reduce the dimensionality of the original high-dimensional feature data. LDA attempts to find a new direction to project all the original data onto, minimizing the intra-class variance of data from the same speaker within this direction, while maximizing the distance between different speakers. This reduces the impact of channel differences and improves the accuracy of voiceprint recognition.

[0089] In some non-limiting embodiments, the extracted identity vector for each registrant typically has a dimension of 400–600, while the dimensionality-reduced identity vector typically has a dimension of 200–300. Length normalization normalizes the dimensionality-reduced identity vector to a unit space, thereby facilitating similarity scoring and recognition in subsequent steps.

[0090] In step S43, a similarity decision threshold is determined based on the scene category of the registered voice of the registrant to which the identity vector with the highest similarity belongs and the scene category of the voice to be identified.

[0091] In practice, during the voiceprint registration stage, scene detection can be performed on multiple registered voice recordings from the same registrant to determine the scene category of each recording. During the voiceprint authentication stage, scene detection can be performed on the voice recording to be recognized to determine its scene category. Typically, multiple registered voice recordings from the same registrant will have the same scene category. If multiple registered voice recordings have different scene categories, the scene category with the highest percentage can be selected as the scene category in which the registrant registered.

[0092] In some non-limiting embodiments, sound source localization or energy-based scene detection and classification methods can be employed. The main purpose is to distinguish the near-field category and signal-to-noise ratio (SNR) level of the currently registered or test speech. For registered speech with a low SNR, the user can be prompted to re-enter. In text-related short speech recognition tasks, such as voiceprint wake-up tasks, the scene detection module can also incorporate keyword recognition; for registered speech that does not match the keywords, the user can be prompted to re-enter.

[0093] In step S44, if the similarity between the identity vector with the highest similarity and the second voiceprint feature vector is greater than or equal to the similarity decision threshold, then it is confirmed that the voice to be identified belongs to the registrant to whom the identity vector with the highest similarity belongs.

[0094] There is a preset mapping relationship between the combination of scene categories of the registered speech and the scene categories of the speech to be recognized, and the similarity judgment threshold. This mapping relationship can be a mapping table obtained through a large number of voiceprint recognition experiments. The specific contents of the mapping table may differ in different application scenarios.

[0095] In this embodiment of the invention, during the voiceprint authentication stage, scene (environment) factors of the registered speech and the speech to be identified are considered, including far-field or near-field, high signal-to-noise ratio or low signal-to-noise ratio, etc. Based on the combination of scene categories of the registered speech and the speech to be identified, a corresponding similarity decision threshold is adaptively selected (for example, the similarity decision threshold corresponding to a certain combination of scene categories can be determined by looking up a preset mapping table). Then, the final voiceprint recognition result is determined based on the comparison result between the highest similarity and the similarity decision threshold. This approach fully considers the influence of scene / environment factors on the similarity calculation result, which can further improve the accuracy of the voiceprint recognition result.

[0096] Reference Figure 5 , Figure 5 This is a schematic diagram of the structure of a voiceprint recognition device according to an embodiment of the present invention. The voiceprint recognition device may include:

[0097] The voiceprint feature vector determination module 51 is used to determine the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant, as well as the posterior probability of each first voiceprint feature vector.

[0098] The weight coefficient determination module 52 is used to determine the weight coefficient of each first voiceprint feature vector based on the posterior probability.

[0099] The registrant identity vector determination module 53 is used to perform a weighted operation on the first voiceprint feature vector of each registered voice based on the determined weight coefficients to obtain the registrant's identity vector;

[0100] The voiceprint recognition module 54 is used to perform voiceprint recognition on the speech to be recognized by using the identity vectors of multiple registrants.

[0101] In specific implementation, the aforementioned voiceprint recognition device may correspond to a chip with voiceprint recognition function; or to a chip module with voiceprint recognition function in a terminal; or to the terminal itself.

[0102] For details regarding the principle, implementation, and beneficial effects of this voiceprint recognition device, please refer to the preceding text. Figures 1 to 4 The descriptions of the voiceprint recognition method shown are not repeated here.

[0103] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the above-described... Figures 1 to 4 The steps of the voiceprint recognition method are shown. The computer-readable storage medium may include non-volatile or non-transitory memory, and may also include optical discs, hard disk drives, solid-state drives, etc.

[0104] Specifically, in this embodiment of the invention, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0105] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0106] This invention also provides a terminal, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor runs the computer program, it performs the above-described... Figures 1 to 4 The steps of the voiceprint recognition method are shown. The terminal may include, but are not limited to, mobile phones, computers, tablets, and other terminal devices, and may also be servers, cloud platforms, etc.

[0107] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or in a combination of hardware and software functional units. For example, for various devices or products applied to or integrated into a chip, each module / unit can be implemented using hardware such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware such as circuits; for various devices or products applied to or integrated into a chip module, each module / unit can be implemented using hardware such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0110] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0111] In the embodiments of this application, "multiple" refers to two or more.

[0112] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0113] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0114] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A voiceprint recognition method, characterized in that, include: Determine the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector; Based on the posterior probability, the weight coefficient of each first voiceprint feature vector is determined, including: comparing the posterior probability of each first voiceprint feature vector with a preset probability threshold; normalizing each posterior probability that is greater than or equal to the probability threshold, and using each normalized posterior probability as the weight coefficient of its respective first voiceprint feature vector. Based on the determined weighting coefficients, the first voiceprint feature vector of each registered voice is weighted to obtain the identity vector of the registrant; The voiceprint recognition of the speech to be identified is performed using the identity vectors of multiple registrants.

2. The method according to claim 1, characterized in that, The step of determining the weight coefficients of each first voiceprint feature vector based on the posterior probability further includes: The weight coefficients of the first voiceprint feature vectors to which each posterior probability is less than the probability threshold are set to 0.

3. The method according to claim 1, characterized in that, The process of using the identity vectors of multiple registered users to perform voiceprint recognition of the speech to be recognized includes: Determine the second voiceprint feature vector of the speech to be recognized; Determine the identity vector that has the highest similarity to the second voiceprint feature vector; The similarity judgment threshold is determined based on the scene category of the registered voice of the registrant to which the identity vector with the highest similarity belongs and the scene category of the voice to be identified; If the similarity between the identity vector with the highest similarity and the second voiceprint feature vector is greater than or equal to the similarity judgment threshold, then the speech to be identified is confirmed to belong to the registrant to which the identity vector with the highest similarity belongs; There is a preset mapping relationship between the combination of the scene category of the registered speech and the scene category of the speech to be recognized and the similarity judgment threshold.

4. The method according to claim 3, characterized in that, The process of determining the identity vector with the highest similarity to the second voiceprint feature vector includes: The probabilistic linear discriminant analysis (PLDA) algorithm is used to calculate the PLDA score between the identity vector and the second voiceprint feature vector for each registrant. The identity vector with the highest PLDA score among the second voiceprint feature vectors is taken as the identity vector with the highest similarity to the second voiceprint feature vector.

5. The method according to claim 3, characterized in that, Before determining the identity vector with the highest similarity to the second voiceprint feature vector, the method further includes: The identity vector and the second voiceprint feature vector of each registrant are dimensionality reduced and then length normalized.

6. The method according to claim 1, characterized in that, The determination of the first voiceprint feature vector of each of the multiple registered voices belonging to the same registrant, and the posterior probability of each first voiceprint feature vector, includes: For each registered speech, determine the posterior distribution of the voiceprint features of the registered speech in the full factor difference space, wherein the mean of the posterior distribution is used as the first voiceprint feature vector of the registered speech. Determine the probability values ​​of the first voiceprint feature vectors of each of the other registered voices, excluding the first voiceprint feature vector of the registered voice, on the posterior distribution; The average of the determined probability values ​​is used as the posterior probability of the first voiceprint feature vector of the registered speech.

7. The method according to claim 6, characterized in that, For each registered speech, the posterior distribution of the speaker characteristics of that registered speech in the full factorial difference space is determined as follows: Acoustic features are extracted for each registered speech to determine the acoustic features of each registered speech. The acoustic features are preprocessed to obtain a frame feature sequence for each registered speech; Based on the frame feature sequence, the posterior distribution of the voiceprint features of the registered speech in the full factorial difference space is determined.

8. A voiceprint recognition device, characterized in that, include: The voiceprint feature vector determination module is used to determine the first voiceprint feature vector of each of multiple registered voices belonging to the same registrant, as well as the posterior probability of each first voiceprint feature vector; The weight coefficient determination module is used to determine the weight coefficient of each first voiceprint feature vector based on the posterior probability, including: comparing the posterior probability of each first voiceprint feature vector with a preset probability threshold; normalizing each posterior probability that is greater than or equal to the probability threshold, and using each normalized posterior probability as the weight coefficient of the corresponding first voiceprint feature vector. The registrant identity vector determination module is used to perform a weighted operation on the first voiceprint feature vector of each registered voice based on the determined weight coefficients to obtain the registrant's identity vector; The voiceprint recognition module is used to perform voiceprint recognition on the speech to be recognized by using the identity vectors of multiple registrants.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the voiceprint recognition method according to any one of claims 1 to 7.

10. A terminal comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the voiceprint recognition method according to any one of claims 1 to 7.

Images