A voiceprint recognition method, device and storage medium
By dynamically adjusting the threshold for voiceprint recognition, the problems of poor threshold generalization ability and inconsistent recognition accuracy in open set voiceprint recognition are solved, achieving higher recognition accuracy and stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD
- Filing Date
- 2023-09-04
- Publication Date
- 2026-06-23
AI Technical Summary
Existing voiceprint recognition technologies suffer from poor threshold generalization and inconsistent recognition accuracy in open-set scenarios, especially with low accuracy when data is imbalanced.
An open-set voiceprint recognition method based on dynamic threshold is adopted. By extracting voiceprint feature vectors from multiple known speakers, a voiceprint feature representation library is established. The dynamic relationship between the number of voiceprint feature vectors and the optimal dynamic threshold is determined, and the threshold is dynamically adjusted to adapt to different scenarios and data imbalances.
It improves the accuracy and stability of voiceprint recognition, can better adapt to different scenarios and data imbalances, and significantly improves the efficiency and accuracy of open set voiceprint recognition.
Smart Images
Figure CN117079654B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a voiceprint recognition method, apparatus, and storage medium. Background Technology
[0002] Voiceprint recognition (VPR) is a biometric technology that automatically identifies a speaker based on the characteristics of their voice. Because the characteristics of a speaker's voice are related to each person's vocal organs, vocal habits, and other factors, these differences result in each speaker's voice possessing unique characteristic representations, thus enabling voiceprint recognition.
[0003] Most voiceprint recognition technologies are designed for closed-set classification scenarios, where the speaker categories in the test set are included in the speaker categories already present in the training set. However, in practical applications, open scenarios are more common, meaning that speakers unknown in the training set may appear in the test set. Therefore, Open Set Recognition (OSR) is needed to solve the classification and recognition problem in open scenarios.
[0004] In open-set voiceprint recognition applications, the choice of the decision threshold is crucial. The threshold is used to measure whether the test voiceprint audio sample matches the category in the training set, that is, to determine whether the test set sample is a known speaker or an unknown speaker. The choice of threshold directly affects the accuracy of open-set voiceprint recognition.
[0005] However, existing methods generally use static thresholds such as fixed thresholds or dual thresholds, but static thresholding methods have two main problems:
[0006] First, since the training set and the test set are often in different collection environments, the generalization ability of the static threshold is poor and may not be able to adapt to all scenarios or all categories, thus affecting the accuracy of open set voiceprint recognition. Second, for each voiceprint category with imbalanced data, the feature vector center obtained from a small number of feature vectors may not be able to fully represent the voiceprint category. Using the static threshold often results in inconsistent recognition accuracy for different categories, especially when the number of feature vectors is small, the recognition accuracy is generally low. Summary of the Invention
[0007] This application provides a voiceprint recognition method, apparatus, and storage medium, aiming to improve the accuracy of voiceprint recognition.
[0008] In a first aspect, embodiments of this application provide an open-set voiceprint recognition method based on dynamic thresholds, the method comprising:
[0009] Extract multiple voiceprint feature vectors corresponding to each known speaker from a pool of known speakers, and establish a voiceprint feature representation library;
[0010] Based on the aforementioned voiceprint feature representation library, determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold;
[0011] Based on the dynamic relationship, determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library;
[0012] Based on the initial optimal dynamic threshold corresponding to each known speaker, it is determined whether the voiceprint test data belongs to a known speaker or an unknown speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated.
[0013] Optionally, multiple voiceprint feature vectors corresponding to each known speaker are extracted from multiple known speakers to establish a voiceprint feature representation library, including:
[0014] The voiceprint recognition model, trained by a deep learning network based on metric learning, extracts the voiceprint feature vector of any known speaker, where each voiceprint feature vector carries the label of the corresponding speaker.
[0015] According to the same label, the multiple voiceprint feature vectors of each known speaker are classified into categories.
[0016] Obtain multiple voiceprint feature vectors corresponding to each of the multiple known speakers, and construct a voiceprint feature representation library.
[0017] Optionally, based on the voiceprint feature representation library, the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold is determined, including:
[0018] Gradually increase the number of voiceprint feature vectors;
[0019] For any number of voiceprint feature vectors, a corresponding number of voiceprint feature vectors are randomly selected from each known speaker to determine the center of the voiceprint feature vectors. The intra-class distance and inter-class distance of each known speaker are calculated to determine the similarity probability distribution. The optimal dynamic threshold corresponding to each known speaker under the given number of voiceprint feature vectors is then determined.
[0020] Optionally, the method further includes:
[0021] For any number of voiceprint feature vectors, the average optimal dynamic threshold corresponding to the multiple known speakers is determined based on the optimal dynamic threshold corresponding to each known speaker under the number of voiceprint feature vectors.
[0022] A threshold adjustment value is determined based on the dynamic change of the number of any voiceprint feature vectors and the average optimal dynamic threshold corresponding to the number of voiceprint feature vectors, wherein the threshold adjustment value is used to adjust the dynamic threshold of each known speaker.
[0023] Optionally, based on the dynamic relationship, the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library is determined, including:
[0024] For any known speaker, based on all the voiceprint feature vectors corresponding to the known speaker, determine the center of the voiceprint feature vectors corresponding to all the voiceprint feature vectors, calculate the intra-class spacing and inter-class spacing of the known speaker, and determine the initial optimal dynamic threshold corresponding to the known speaker.
[0025] Optionally, determining whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each known speaker includes:
[0026] Extract the voiceprint feature test vector corresponding to the voiceprint test data;
[0027] For any known speaker, calculate the distance between the voiceprint feature test vector and the center of the voiceprint feature vector corresponding to the known speaker, and determine whether the voiceprint test data belongs to the known speaker based on the distance and the initial optimal dynamic threshold corresponding to the known speaker.
[0028] Optionally, the method further includes:
[0029] When the voiceprint test data belongs to any known speaker, the voiceprint feature test vector corresponding to the voiceprint test data is added to the category corresponding to the known speaker in the voiceprint feature representation library;
[0030] When the voiceprint test data does not belong to any known speaker, a new known speaker is added to the voiceprint feature representation library, and the voiceprint feature vector corresponding to the voiceprint test data is added to the category of the new known speaker in the voiceprint feature representation library.
[0031] Optionally, the method further includes:
[0032] When the number of voiceprint feature vectors in the voiceprint feature representation library changes, the number of all voiceprint feature vectors corresponding to each known speaker is counted, and the initial optimal dynamic threshold corresponding to each known speaker is updated according to the threshold adjustment value.
[0033] Secondly, embodiments of this application provide an open-set voiceprint recognition device based on dynamic thresholds, the device comprising:
[0034] The voiceprint feature representation library construction module is used to extract multiple voiceprint feature vectors corresponding to each known speaker from multiple known speakers and establish a voiceprint feature representation library.
[0035] The dynamic relationship determination module is used to determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold based on the voiceprint feature representation library.
[0036] The initial optimal dynamic threshold determination module is used to determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library based on the dynamic relationship.
[0037] The voiceprint recognition module is used to determine whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each known speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated.
[0038] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the open-set voiceprint recognition method based on dynamic thresholds as described in the first aspect of the embodiments.
[0039] Beneficial effects:
[0040] Multiple voiceprint feature vectors corresponding to each known speaker are extracted from multiple known speakers to establish a voiceprint feature representation library. Then, the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold is determined. Based on the dynamic relationship, the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library is determined.
[0041] During the identification process, based on the initial optimal dynamic threshold corresponding to each known speaker, it is determined whether the voiceprint test data belongs to a known speaker or an unknown speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated.
[0042] This method determines an initial optimal dynamic threshold for each known speaker based on the relationship between the number of voiceprint feature vectors and the optimal threshold. Furthermore, it can automatically adjust the initial optimal dynamic threshold for each known speaker according to the subsequent changes in the number of voiceprint feature vectors. This not only enables voiceprint recognition to better adapt to imbalanced data and various different scenarios, but also significantly improves the accuracy and stability of open set voiceprint recognition. Attached Figure Description
[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0044] Figure 1 This is a flowchart of the steps of an open set voiceprint recognition method based on dynamic threshold proposed in an embodiment of this application;
[0045] Figure 2 This is a similarity probability distribution diagram proposed in an embodiment of this application;
[0046] Figure 3 This is a schematic diagram showing the variation of the number of voiceprint feature vectors with the optimal dynamic threshold according to an embodiment of this application;
[0047] Figure 4 This is a functional block diagram of an open set voiceprint recognition device based on dynamic threshold proposed in an embodiment of this application. Detailed Implementation
[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] In open-set voiceprint recognition applications, a threshold is used to measure whether the test voiceprint audio sample matches the category in the training set, that is, to determine whether the test set sample is a known speaker or an unknown speaker. The choice of threshold directly affects the accuracy of open-set voiceprint recognition. Existing methods generally use static thresholds such as fixed thresholds or dual thresholds. However, static thresholds have poor generalization ability and may not be suitable for all scenarios or all categories. Furthermore, for voiceprint categories with imbalanced data, the feature vector centers obtained from a small number of feature vectors may not fully represent the voiceprint category. Using static thresholds often results in inconsistent recognition accuracy for different categories, especially when the number of feature vectors is small, the recognition accuracy is generally low. In order to improve the accuracy of voiceprint recognition, this application provides an open-set voiceprint recognition method based on dynamic thresholds.
[0050] Reference Figure 1 The diagram illustrates a flowchart of an open-set voiceprint recognition method based on dynamic thresholds according to an embodiment of this application. The method specifically includes the following steps:
[0051] S101: Extract multiple voiceprint feature vectors corresponding to each known speaker from multiple known speakers, and establish a voiceprint feature representation library.
[0052] In one feasible implementation, a voiceprint recognition model trained by a deep learning network based on metric learning can be used to extract the voiceprint feature vector of any known speaker. For example, a deep learning network is built based on deep metric learning, and then trained on a large training set of voiceprints of known speakers to obtain a closed-set voiceprint recognition model with good convergence and high accuracy. The backbone network of the voiceprint recognition model is used as a feature extractor to extract features from the voiceprint data and obtain a high-dimensional voiceprint feature vector.
[0053] Furthermore, each voiceprint feature vector carries a label corresponding to the speaker. Multiple voiceprint feature vectors of each known speaker can be classified according to the same label to obtain multiple voiceprint feature vectors corresponding to each of the multiple known speakers, and a voiceprint feature representation library can be constructed. The voiceprint feature vectors of each known speaker in the voiceprint feature representation library belong to one class.
[0054] S102: Based on the voiceprint feature representation library, determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold.
[0055] For each known speaker category, the number of voiceprint feature vectors included will affect the feature vector center of that category. When determining whether a voiceprint feature vector belongs to a certain category, it is necessary to determine the distance between the voiceprint feature vector and the feature vector center of different categories. If the distance is less than the threshold of a category, the voiceprint feature vector can be considered to belong to that category. Therefore, the choice of threshold is closely related to the number of voiceprint feature vectors. Thus, in this embodiment, the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold is first determined in the voiceprint feature representation library.
[0056] Specifically, the number of voiceprint feature vectors is gradually increased. For any number of voiceprint feature vectors, a corresponding number of voiceprint feature vectors are randomly selected from each known speaker to determine the center of the voiceprint feature vectors. The intra-class distance and inter-class distance of each known speaker are calculated to determine the similarity probability distribution and to determine the optimal dynamic threshold for each known speaker under the given number of voiceprint feature vectors.
[0057] For example, we can first set the initial number of voiceprint feature vectors ζ0, the first level of refinement σ, and the number of extractions β, and then gradually increase the number of voiceprint feature vectors ζ through r1, where,
[0058] ζ=ζ0+r1*σ,r1=1,2,…,
[0059] For each number of voiceprint feature vectors ζ, randomly select ζ voiceprint feature vectors for each class from the voiceprint feature representation library, and calculate the voiceprint feature vector center of each class of ζ voiceprint feature vectors.
[0060] After determining the center of the voiceprint feature vector for each known speaker or for each class of ζ voiceprint feature vectors, the intra-class distance and inter-class distance for each class are calculated. The intra-class distance is the distance from any voiceprint feature vector of that class to the center of the voiceprint feature vector of that class, and the inter-class distance is the distance from any voiceprint feature vector of a different class to the center of the voiceprint feature vector of that class.
[0061] In practical implementation, when calculating the distance from the high-dimensional voiceprint feature vector to the center of the voiceprint feature vector, either Euclidean distance or cosine distance can be used. This embodiment does not impose any restrictions. For example:
[0062] The Euclidean distance calculation formula is:
[0063]
[0064] The formula for calculating cosine distance is:
[0065]
[0066] Where dist(X,Y) represents the Euclidean distance, cos(θ) represents the cosine distance, n represents the dimension of the eigenvector, and x... i y i Let X and Y represent the i-th dimension of the two voiceprint feature vectors X and Y, respectively.
[0067] In one feasible implementation, an initial threshold α, a second level of refinement f, and a number of steps r2 can be set. By gradually increasing the threshold k = α + r2 * f, r2 = 1, 2, ..., the resolution of each category at different thresholds is determined for any number of voiceprint feature vectors ζ. Based on the optimal resolution, the optimal dynamic threshold for each category when the number of voiceprint feature vectors ζ is determined, where the optimal dynamic threshold for the nth category with the number of voiceprint feature vectors ζ is denoted as...
[0068] In this embodiment, in order to adaptively adjust the threshold in the future, for any number of voiceprint feature vectors, the average optimal dynamic threshold corresponding to the multiple known speakers is determined based on the optimal dynamic threshold corresponding to each known speaker under that number of voiceprint feature vectors; based on the dynamic change of the average optimal dynamic threshold corresponding to the number of any voiceprint feature vectors, a threshold adjustment value is determined, wherein the threshold adjustment value is used to adjust the dynamic threshold of each known speaker.
[0069] For example, when the number of voiceprint feature vectors is ζ, the optimal dynamic threshold for the nth class is denoted as . The average optimal dynamic threshold for all N categories is:
[0070]
[0071] In one feasible implementation, the number of voiceprint feature vectors ζ and the average optimal dynamic threshold T can be plotted. ζ The dynamic change curve is obtained, and then its change pattern is captured by statistical methods to obtain the threshold adjustment value for adjusting the dynamic threshold of each class: ΔT=f(ζ).
[0072] S103: Based on the dynamic relationship, determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library.
[0073] For any known speaker, based on all the voiceprint feature vectors corresponding to that known speaker, determine the center of the voiceprint feature vectors corresponding to all the voiceprint feature vectors, and calculate the intra-class distance and inter-class distance of that known speaker. The method for calculating the distance can be the Euclidean distance or cosine distance mentioned above. Then, determine the initial optimal dynamic threshold T corresponding to that known speaker. For example, the initial optimal dynamic threshold T for each class can be selected using the probability distribution histogram and the corresponding recognition rate curve.
[0074] S104: Determine whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each known speaker.
[0075] After constructing the voiceprint feature representation library, the first step is to determine whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each category.
[0076] Specifically, when determining whether the voiceprint test data belongs to a known speaker or an unknown speaker, the voiceprint feature test vector corresponding to the voiceprint test data is first extracted. A pre-trained voiceprint recognition model can be used, with the backbone network of the voiceprint recognition model serving as a feature extractor. The feature extractor is then used to extract the voiceprint feature test vector corresponding to the voiceprint test data.
[0077] For any known speaker, or for each class, the distance between the voiceprint feature test vector and the center of the voiceprint feature vector of that class is calculated. Then, based on the distance and the initial optimal dynamic threshold corresponding to the known speaker, it is determined whether the voiceprint test data belongs to the known speaker.
[0078] For example, the initial optimal dynamic threshold for the first known speaker is T1, and the initial optimal dynamic threshold for the second known speaker is T2. The distance from the voiceprint feature test vector to the center of the voiceprint feature vector for the first known speaker is denoted as dist1, and the distance from the voiceprint feature test vector to the center of the voiceprint feature vector for the second known speaker is denoted as dist2. If dist1 is less than T1, then the voiceprint test data is the voice data of the first known speaker; if dist2 is less than T2, then the voiceprint test data is the voice data of the second known speaker.
[0079] In one feasible implementation, the method also includes a dynamic update mechanism for the voiceprint feature representation library and dynamic adjustments to the threshold for each class to improve the accuracy of voiceprint recognition.
[0080] Specifically, when the voiceprint test data belongs to any known speaker, the voiceprint feature test vector corresponding to the voiceprint test data is added to the category corresponding to the known speaker in the voiceprint feature representation library; when the voiceprint test data does not belong to any known speaker, a new known speaker is added to the voiceprint feature representation library, and the voiceprint feature vector corresponding to the voiceprint test data is added to the category of the new known speaker in the voiceprint feature representation library.
[0081] When the number of voiceprint feature vectors in the voiceprint feature representation library changes, the number of all voiceprint feature vectors corresponding to each known speaker is counted, and the initial optimal dynamic threshold corresponding to each known speaker is updated according to the threshold adjustment value.
[0082] For example, the formula is adjusted as follows:
[0083] T new =(ΔT±1)*T=(f(ζ)±1)*T
[0084] The optimal dynamic threshold T after updating any category new This serves as the threshold for the next round of voiceprint recognition.
[0085] In this embodiment, an initial optimal dynamic threshold is determined for each class based on the number of voiceprint feature vectors of each known speaker. This allows voiceprint recognition to maintain excellent performance under different speakers and even in different environments and scenarios, greatly improving the accuracy and adaptability of voiceprint recognition. Furthermore, the threshold can be dynamically adjusted, automatically adjusting the threshold corresponding to each class based on changes in the number of voiceprint feature vectors. By dynamically adjusting the threshold, the voiceprint recognition system can not only better adapt to imbalanced data and various different scenarios, but also significantly improve the efficiency and stability of open set voiceprint recognition.
[0086] In one feasible implementation, the application scenario for voiceprint recognition is as follows:
[0087] A1: Construct the training and test sets.
[0088] Specifically, based on the Aidatatang_200zh Chinese audio dataset, VAD (Voice Activity Detection) endpoint recognition was performed, and pure human voice audio datasets of 400 people were obtained. Each person's total duration exceeded 300 seconds, resulting in an audio dataset of approximately 60 hours.
[0089] Then, 300 people were selected as known speakers and 100 as unknown speakers. The first 300 seconds of the 300 known speakers were used as the training set, and the remaining audio of the 300 known speakers and the audio of the 100 unknown speakers were used as the open set test set. That is, the open set test set includes the audio data of the known speakers and the audio data of the unknown speakers.
[0090] The audio data in both the training and testing sets were cut into 3-second audio samples. The training and validation sets consisted of 300 people, while the validation set was 30% randomly selected from the training set and did not participate in the direct training.
[0091] A2: Construct a voiceprint feature representation library.
[0092] A Circle Loss-based metric learning model is trained using the training set to obtain a voiceprint recognition model. This model is then used to extract voiceprint feature vectors from the training set. Each voiceprint feature vector carries a label for the speaker. These voiceprint feature vectors can be denoted as... Let N represent the i-th voice feature vector of the N-th speaker, where N all =300, i = 1, 2, ..., n, where n is the total number of feature vectors for each speaker. In this embodiment, n = 70, meaning there are a total of 300 speaker categories, with 70 voice feature vectors for each category. In this embodiment, the dimension of the voice feature vectors is uniformly set to M = 512. The feature vectors are then stored according to their corresponding speaker labels to establish a voiceprint feature representation library.
[0093] A3: Determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold.
[0094] Specifically, each time, ζ voiceprint feature vectors (0 < ζ ≤ 70) are randomly selected from the voiceprint feature representation database for each type of speaker.
[0095] Then, the centers of the ζ voiceprint feature vectors for each speaker type are calculated:
[0096]
[0097] in, The M-th dimension of the voiceprint feature vector of the i-th speaker of the N-th class is represented.
[0098] Next, calculate the intra-class spacing and inter-class spacing for each class, using X. 0 The voiceprint feature vector representing the same speaker is represented by X. 1 Voiceprint feature vectors representing different speakers.
[0099] Taking Euclidean distance as an example, the calculation process for intra-class spacing is as follows:
[0100]
[0101] Among them, X_center N Let X be the center of the ζ voiceprint feature vectors of the Nth speaker. 0,1 M is the first voiceprint feature vector of the Nth type speaker; M is the dimension of the feature vector, and in this embodiment, M = 512.
[0102] The calculation process for inter-class spacing is similar; by calculating X... 0 With X 1 The similarity probability distribution is used to obtain the optimal threshold α corresponding to the ζ voiceprint feature vectors. ACC＝max , set the optimal threshold α ACC The optimal dynamic threshold for the nth class with the number of voiceprint feature vectors ζ is denoted as .
[0103] Reference Figure 2 The diagram shows a similarity probability distribution provided in an embodiment of this application. Figure 2 In the diagram, the horizontal axis represents the threshold α, the left vertical axis represents the accuracy, and the right vertical axis represents the accuracy. Since the threshold essentially reflects the distance between the centers of the voiceprint feature vectors, the number of voiceprint feature vectors with a distance of the threshold is determined under different thresholds, and the accuracy of voiceprint recognition is also different under different thresholds.
[0104] Therefore, for any class, we can plot the distribution of voiceprint feature vectors in the closed set (consisting of known speakers' data) and the open set (consisting of known speakers and unknown speakers) under different thresholds for a given threshold α. Under the optimal distribution, we can plot the closed set accuracy curve, the open set accuracy curve, and the total accuracy curve. We can choose the intersection of the closed set accuracy curve, the open set accuracy curve, and the total accuracy curve as the optimal threshold for that class. Alternatively, we can choose the threshold corresponding to the highest total accuracy curve, i.e., the value with the largest total accuracy, as the optimal dynamic threshold for that class. For example, we can choose the optimal dynamic threshold as 0.847, at which point the maximum total accuracy is 0.86.
[0105] Furthermore, the number ζ of voiceprint feature vectors included in the statistical voiceprint feature vector center calculation is compared with the optimal dynamic threshold. Draw histograms and accuracy change curves to illustrate the changing patterns.
[0106] Reference Figure 3 This diagram illustrates the relationship between the number of voiceprint feature vectors and the optimal dynamic threshold provided in the embodiments of this application. Figure 3The horizontal axis represents the number of voiceprint feature vectors ζ, the left vertical axis represents accuracy, and the right vertical axis represents the optimal dynamic threshold. When the number of voiceprint feature vectors ζ is larger, the optimal dynamic threshold changes less and the accuracy tends to be stable. The adjustment law of the dynamic threshold can be obtained from the change law graph, thereby solving the problem of threshold versatility under data imbalance and inconsistent scenarios. In practical applications, the corresponding change law graph can be drawn according to different data.
[0107] A4: Calculate the center X_center of the voiceprint feature vectors for all voiceprint feature vectors of each speaker class in the voiceprint feature representation library. N With the initial optimal dynamic threshold
[0108] A5: Extract the voiceprint feature vector from the voiceprint test data in the test set based on the voiceprint recognition model, denoted as Y. 0,1 Then calculate Y 0,1 X_center of each type of voiceprint feature vector in the voiceprint feature representation library N similarity dist(Y) 0,1 ,X_center N ), and according to Determine whether the voiceprint test data belongs to a known speaker or an unknown speaker of that type.
[0109] A6: If the speaker category is known, add the category of known speakers in the voiceprint feature representation library and update the number of voiceprint feature vectors of that category, thereby updating the optimal dynamic threshold of that category; if the speaker category is unknown, create a new category N+1 in the voiceprint feature representation library and calculate the optimal dynamic threshold corresponding to category N+1, thereby forming a dynamic threshold adjustment.
[0110] A7: Perform voiceprint recognition based on the updated optimal dynamic threshold for each category, and repeat steps A5-A6 to complete the open set voiceprint recognition test under dynamic threshold adjustment.
[0111] Reference Figure 4 The diagram illustrates a functional block diagram of an open-set voiceprint recognition device based on a dynamic threshold according to an embodiment of this application. The device includes:
[0112] The voiceprint feature representation library construction module 100 is used to extract multiple voiceprint feature vectors corresponding to each known speaker from multiple known speakers and establish a voiceprint feature representation library.
[0113] The dynamic relationship determination module 200 is used to determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold based on the voiceprint feature representation library.
[0114] The initial optimal dynamic threshold determination module 300 is used to determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library according to the dynamic relationship.
[0115] The voiceprint recognition module 400 is used to determine whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each known speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated.
[0116] Optionally, the voiceprint feature representation library construction module includes:
[0117] The feature extraction unit is used to extract the voiceprint feature vector of any known speaker by using a voiceprint recognition model trained by a deep learning network built on metric learning. Each voiceprint feature vector carries the label of the corresponding speaker.
[0118] The classification unit is used to classify multiple voiceprint feature vectors of each known speaker according to the same label;
[0119] The construction unit is used to obtain multiple voiceprint feature vectors corresponding to each of the multiple known speakers and construct a voiceprint feature representation library.
[0120] Optionally, the dynamic relationship determination module includes:
[0121] Add units to gradually increase the number of voiceprint feature vectors;
[0122] The optimal dynamic threshold determination unit is used to randomly select a corresponding number of voiceprint feature vectors from each known speaker for any number of voiceprint feature vectors, determine the center of the voiceprint feature vectors, calculate the intra-class distance and inter-class distance of each known speaker, determine the similarity probability distribution, and determine the optimal dynamic threshold corresponding to each known speaker under the number of voiceprint feature vectors.
[0123] Optionally, the device further includes:
[0124] The average optimal dynamic threshold determination unit is used to determine the average optimal dynamic threshold corresponding to the multiple known speakers for any number of voiceprint feature vectors, based on the optimal dynamic threshold corresponding to each known speaker for that number of voiceprint feature vectors.
[0125] The threshold adjustment value determination unit is used to determine the threshold adjustment value based on the dynamic change of the number of any voiceprint feature vectors and the average optimal dynamic threshold corresponding to the number of voiceprint feature vectors, wherein the threshold adjustment value is used to adjust the dynamic threshold of each known speaker.
[0126] Optionally, the initial optimal dynamic threshold determination module includes:
[0127] The initial optimal dynamic threshold determination unit is used to determine the center of the voiceprint feature vector corresponding to all voiceprint feature vectors corresponding to any known speaker, and to calculate the intra-class distance and inter-class distance of the known speaker, and determine the initial optimal dynamic threshold corresponding to the known speaker.
[0128] Optionally, the voiceprint recognition module includes:
[0129] The test extraction unit is used to extract the voiceprint feature test vector corresponding to the voiceprint test data;
[0130] The identification unit is used to calculate, for any known speaker, the distance between the voiceprint feature test vector and the center of the voiceprint feature vector corresponding to the known speaker, and determine whether the voiceprint test data belongs to the known speaker based on the distance and the initial optimal dynamic threshold corresponding to the known speaker.
[0131] Optionally, the device further includes:
[0132] The first update module is used to add the voiceprint feature test vector corresponding to the voiceprint test data to the category corresponding to the known speaker in the voiceprint feature representation library when the voiceprint test data belongs to any known speaker;
[0133] The second update module is used to add a new known speaker to the voiceprint feature representation library when the voiceprint test data does not belong to any known speaker, and to add the voiceprint feature vector corresponding to the voiceprint test data to the category of the new known speaker in the voiceprint feature representation library.
[0134] Optionally, the device further includes:
[0135] The threshold update module is used to count the number of all voiceprint feature vectors corresponding to each known speaker when the number of voiceprint feature vectors in the voiceprint feature representation library changes, and update the initial optimal dynamic threshold corresponding to each known speaker according to the threshold adjustment value.
[0136] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the open-set voiceprint recognition method based on dynamic thresholds as described in the embodiments of the present application.
[0137] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0139] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0142] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.
[0143] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0144] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. An open-set voiceprint recognition method based on dynamic thresholding, characterized in that, The method includes: Extract multiple voiceprint feature vectors corresponding to each known speaker from a pool of known speakers, and establish a voiceprint feature representation library; Based on the aforementioned voiceprint feature representation library, determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold; Based on the dynamic relationship, determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library; Based on the initial optimal dynamic threshold corresponding to each known speaker, it is determined whether the voiceprint test data belongs to a known speaker or an unknown speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated. Specifically, based on the voiceprint feature representation library, determining the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold includes: Gradually increase the number of voiceprint feature vectors; For any number of voiceprint feature vectors, a corresponding number of voiceprint feature vectors are randomly selected from each known speaker to determine the center of the voiceprint feature vectors. The intra-class distance and inter-class distance of each known speaker are calculated to determine the similarity probability distribution and the optimal dynamic threshold corresponding to each known speaker under the given number of voiceprint feature vectors is determined.
2. The method according to claim 1, characterized in that, Extract multiple voiceprint feature vectors corresponding to each known speaker from a pool of known speakers, and establish a voiceprint feature representation library, including: The voiceprint recognition model, trained by a deep learning network based on metric learning, extracts the voiceprint feature vector of any known speaker, where each voiceprint feature vector carries the label of the corresponding speaker. The multiple voiceprint feature vectors of each known speaker are categorized according to the same label; Obtain multiple voiceprint feature vectors corresponding to each of the multiple known speakers, and construct a voiceprint feature representation library.
3. The method according to claim 1, characterized in that, The method further includes: For any number of voiceprint feature vectors, the average optimal dynamic threshold corresponding to the multiple known speakers is determined based on the optimal dynamic threshold corresponding to each known speaker under the number of voiceprint feature vectors. A threshold adjustment value is determined based on the dynamic change of the number of any voiceprint feature vectors and the average optimal dynamic threshold corresponding to the number of voiceprint feature vectors, wherein the threshold adjustment value is used to adjust the dynamic threshold of each known speaker.
4. The method according to claim 1, characterized in that, Based on the dynamic relationship, the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library is determined, including: For any known speaker, based on all the voiceprint feature vectors corresponding to the known speaker, determine the center of the voiceprint feature vectors corresponding to all the voiceprint feature vectors, calculate the intra-class spacing and inter-class spacing of the known speaker, and determine the initial optimal dynamic threshold corresponding to the known speaker.
5. The method according to claim 1, characterized in that, Based on the initial optimal dynamic threshold corresponding to each known speaker, the voiceprint test data is determined to belong to a known speaker or an unknown speaker, including: Extract the voiceprint feature test vector corresponding to the voiceprint test data; For any known speaker, calculate the distance between the voiceprint feature test vector and the center of the voiceprint feature vector corresponding to the known speaker, and determine whether the voiceprint test data belongs to the known speaker based on the distance and the initial optimal dynamic threshold corresponding to the known speaker.
6. The method according to claim 1, characterized in that, The method further includes: When the voiceprint test data belongs to any known speaker, the voiceprint feature test vector corresponding to the voiceprint test data is added to the category corresponding to the known speaker in the voiceprint feature representation library; When the voiceprint test data does not belong to any known speaker, a new known speaker is added to the voiceprint feature representation library, and the voiceprint feature vector corresponding to the voiceprint test data is added to the category of the new known speaker in the voiceprint feature representation library.
7. The method according to claim 3, characterized in that, The method further includes: When the number of voiceprint feature vectors in the voiceprint feature representation library changes, the number of all voiceprint feature vectors corresponding to each known speaker is counted, and the initial optimal dynamic threshold corresponding to each known speaker is updated according to the threshold adjustment value.
8. An open-set voiceprint recognition device based on dynamic threshold, characterized in that, The device includes: The voiceprint feature representation library construction module is used to extract multiple voiceprint feature vectors corresponding to each known speaker from multiple known speakers and establish a voiceprint feature representation library. The dynamic relationship determination module is used to determine the dynamic relationship between the number of different voiceprint feature vectors and the optimal dynamic threshold based on the voiceprint feature representation library. The initial optimal dynamic threshold determination module is used to determine the initial optimal dynamic threshold corresponding to all voiceprint feature vectors of each known speaker in the voiceprint feature representation library based on the dynamic relationship. The voiceprint recognition module is used to determine whether the voiceprint test data belongs to a known speaker or an unknown speaker based on the initial optimal dynamic threshold corresponding to each known speaker. When the voiceprint test data belongs to a known speaker, the voiceprint feature vector of the voiceprint test data is added to the voiceprint feature representation library, and the initial optimal dynamic threshold corresponding to the known speaker is updated. The dynamic relationship determination module includes: Add units to gradually increase the number of voiceprint feature vectors; The optimal dynamic threshold determination unit is used to randomly select a corresponding number of voiceprint feature vectors from each known speaker for any number of voiceprint feature vectors, determine the center of the voiceprint feature vectors, calculate the intra-class distance and inter-class distance of each known speaker, determine the similarity probability distribution, and determine the optimal dynamic threshold corresponding to each known speaker under the number of voiceprint feature vectors.
9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the open-set voiceprint recognition method based on dynamic thresholds as described in any one of claims 1 to 7.