A zero-configuration adaptive speaker recognition method and system

Abstract

The application discloses a zero-configuration adaptive speaker recognition method and system, and relates to the technical field of speech signal processing.The application receives an audio stream, performs voice activity detection, acquires a single-person voice segment, extracts a voiceprint embedding vector of the single-person voice segment, performs online clustering, generates a speaker identity pool, calculates a multi-dimensional fusion similarity between a remaining single-person voice segment and a voice model to update the identity pool, and outputs a temporary identity label corresponding to the single-person voice segment.The speaker recognition method provided by the application does not need to pre-register a voiceprint template, and the flexibility and real-time performance of the system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech signal processing, in particular to a zero-configuration adaptive speaker recognition method and system. BACKGROUND

[0002] In scenarios such as multi-person meetings, interviews, customer service quality inspection, etc. that require speech transcription recording, accurately distinguishing "who said what at what time" is a crucial core requirement. Traditional speaker recognition technology usually relies on pre-registered voiceprint templates, which requires collecting user voice data during the system initialization phase, resulting in high deployment costs and poor flexibility.

[0003] Currently, the traditional speaker recognition solution is some open-source solution based on clustering algorithm, which needs to obtain complete and long dialogue audio first, and then performs global analysis and clustering on all speech segments in the entire audio to finally divide different speakers. The main disadvantages of this solution include: unable to meet the real-time requirement. For scenarios that require real-time generation of subtitles with speaker roles or meeting minutes, this "post-analysis" method is useless, and users need to wait for a long processing time after the entire meeting ends to see the results.

[0004] In summary, the existing technology cannot provide a speaker recognition solution that can respond in real time and automatically adapt to dynamic changes in speakers, which greatly hinders the application of speaker recognition technology in more extensive real scenarios. SUMMARY

[0005] The purpose of the present application is to provide a zero-configuration adaptive speaker recognition method and system, aiming to solve the problems of low processing efficiency, high training cost, and inability to realize real-time speaker recognition in traditional methods.

[0006] The first aspect of the embodiment of the present application provides a zero-configuration adaptive speaker recognition method, comprising the following steps:

[0007] receiving an audio stream;

[0008] performing voice activity detection on the audio stream to divide the audio stream into a plurality of single-person speech segments;

[0009] extracting a preset number of single-person speech segments as a first sample set, and the remaining single-person speech segments as a second sample set;

[0010] obtaining a voiceprint embedding vector corresponding to each single-person speech segment in the first sample set, performing online clustering on the voiceprint embedding vector, and generating a speaker identity pool, the speaker identity pool including a plurality of temporary identity labels and speech models corresponding to the temporary identity labels;

[0011] calculate a multi-dimensional fusion similarity of each single-person voice segment in the second sample set and the voice model;

[0012] update the speaker identity pool according to the multi-dimensional fusion similarity, and output a temporary identity label corresponding to the single-person voice segment.

[0013] In other embodiments of the present application, calculating a multi-dimensional fusion similarity of each single-person voice segment in the second sample set and the voice model includes:

[0014] obtaining multi-dimensional information of the single-person voice segment, the multi-dimensional information including a voiceprint embedding vector, a time sequence continuity vector, an emotional state vector, and a text content coherence vector;

[0015] determining weights of the dimensional information in the multi-dimensional information through a fusion decision function;

[0016] calculating a multi-dimensional fusion similarity of the single-person voice segment and each voice model according to the weights of the dimensional information.

[0017] Further, the fusion decision function adopts at least one of a weighted average value, a Bayesian network, or a neural network model, and the weights are adaptively optimized according to the recognition result of the single-person voice segment.

[0018] Further, the voiceprint embedding vector is clustered online to generate a speaker identity pool, including:

[0019] determining the number of clustering clusters through an unsupervised clustering algorithm, and dividing the voiceprint embedding vector into the clustering clusters;

[0020] generating a voice model according to the centroid of the voiceprint embedding vector in the clustering cluster, and assigning a temporary identity label to each clustering cluster.

[0021] Further, the method of determining the number of clustering clusters includes at least one of:

[0022] determining the number of clustering clusters through a clustering effectiveness index;

[0023] adaptively determining the number of clustering clusters through density analysis;

[0024] dynamically incrementally correcting the number of preset clustering clusters.

[0025] In other embodiments of the present application, updating the speaker identity pool according to the multi-dimensional fusion similarity includes:

[0026] when the multi-dimensional fusion similarity is greater than or equal to a preset threshold, determining a temporary identity label corresponding to the single-person voice segment, and updating a voice model corresponding to the temporary identity label based on a voiceprint embedding vector corresponding to the single-person voice segment at a preset learning rate;

[0027] When the multi-dimensional fusion similarity is less than a preset threshold, a new temporary identity label and a new speech model are created to join a speaker identity pool.

[0028] Further, the method further comprises the following steps:

[0029] A user interface is provided to allow an operator to perform at least one of the following operations:

[0030] Renaming or binding the temporary identity label;

[0031] Correcting the temporary identity label corresponding to the single-person speech segment;

[0032] Merging multiple temporary identity labels;

[0033] Forcing an update of the speech model in the speaker identity pool to generate an optimized model.

[0034] Further, a preset number of single-person speech segments are extracted as a first sample set, and the remaining single-person speech segments are extracted as a second sample set, comprising:

[0035] When the number of extracted single-person speech segments reaches the preset number, the extracted single-person speech segments are taken as the first sample set, and the subsequently extracted single-person speech segments are taken as the second sample set.

[0036] Alternatively, a preset number of single-person speech segments are randomly extracted as the first sample set, and the remaining single-person speech segments are taken as the second sample set.

[0037] Further, a deep neural network model is used to obtain a voiceprint embedding vector corresponding to each single-person speech segment in the first sample set, and model parameters of the deep neural network model are dynamically adjusted in the identification process.

[0038] A second aspect of an embodiment of the application provides a zero-configuration adaptive speaker recognition method system, comprising:

[0039] An audio acquisition module is configured to receive an audio stream.

[0040] A voice activity detection module is configured to perform voice activity detection on the audio stream and divide the audio stream into multiple single-person speech segments.

[0041] A sample set division module is configured to extract a preset number of single-person speech segments as a first sample set and the remaining single-person speech segments as a second sample set.

[0042] A voiceprint embedding extraction module is configured to obtain a voiceprint embedding vector corresponding to each single-person speech segment in the first sample set.

[0043] an online clustering module, configured to perform online clustering on the voiceprint embedding vectors to generate a speaker identity pool, the speaker identity pool comprising a plurality of temporary identity labels and voice models corresponding to the temporary identity labels;

[0044] a multi-dimension similarity calculation module, configured to calculate multi-dimension fusion similarity between each single-person voice segment in the second sample set and the voice models;

[0045] an identity pool updating module, configured to update the speaker identity pool according to the multi-dimension fusion similarity, and output the temporary identity label corresponding to the single-person voice segment.

[0046] Compared with the prior art, the embodiments of the present application have the beneficial effects that: the present application does not require the user to pre-register the voiceprint template in the system initialization stage. The system automatically extracts the voiceprint embedding vector from the real-time voice stream, and dynamically generates the temporary identity label and the voice model through the online clustering algorithm to construct the identity pool. The present application does not require pre-registration, reduces the difficulty of system deployment and the user operation burden, has low resource consumption, can be used at any time, can be better applied to temporary users or dynamic scenarios, thereby expanding the flexibility of the application scenarios of speaker recognition. At the same time, the present application continuously trains through human-computer collaboration, not only depends on the voiceprint embedding vector for speaker recognition, but also fuses multi-dimension context information such as time sequence continuity, emotional state, and text coherence to calculate the comprehensive similarity, which significantly improves the recognition robustness in complex environments, and has stronger adaptability to short voice segments and emotional change scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 a flowchart of a zero-configuration adaptive speaker recognition method provided by the present application;

[0048] Figure 2 a flowchart of generating a speaker identity pool provided by the present application;

[0049] Figure 3 a flowchart of calculating multi-dimension fusion similarity between each single-person voice segment in the second sample set and the voice models provided by the present application;

[0050] Figure 4 a flowchart of an embodiment of a zero-configuration adaptive speaker recognition method provided by the present application;

[0051] Figure 5 a structural diagram of a zero-configuration adaptive speaker recognition system provided by the embodiments of the present application. DETAILED DESCRIPTION

[0052] In order to make the technical problems, technical solutions and beneficial effects of the present application clearer, the present application will be further described in detail below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0053] For speaker recognition technology in multi-person conversation scenarios, the current mainstream technical solutions mainly have the following three technical bottlenecks:

[0054] Voiceprint pre-registration dependency problem: existing commercial systems generally use voiceprint pre-registration mechanism, which requires pre-acquisition of specific / free voice samples of target speakers, and establishes a voiceprint database bound to user identity through voiceprint feature extraction. First, additional user registration process is needed in the implementation link, which reduces the system availability; second, in an open dialogue scenario (such as temporary participants or unknown customers calling), the system completely loses the recognition function due to the lack of pre-stored voiceprint data.

[0055] Some open source solutions use offline clustering algorithms, which require complete dialogue recording to be obtained, and then realize speaker differentiation through global voice segment clustering analysis. This solution does not have real-time processing, and cannot meet the timeliness requirements of conference real-time caption generation, and users must bear the processing waiting time after the recording is completed.

[0056] Some specific recognition algorithms have technical limitations on the number of preset speakers, which forcibly classify all voice segments into a preset fixed number of speakers. When the actual number of speakers changes dynamically (such as personnel increase or decrease during the meeting), the system cannot automatically adjust the recognition strategy, resulting in incorrect classification of new speaker voice.

[0057] The existing technical solutions have insurmountable technical obstacles in real-time, adaptability and scene adaptability, which seriously restrict the popularization and application of the technology in real scenarios.

[0058] Figure 1 A flowchart of a zero-configuration adaptive speaker recognition method provided by the first aspect of the present application is shown, which is described in detail as follows:

[0059] S1, receiving an audio stream.

[0060] It should be noted that the method provided by the present application can receive continuous audio input stream in real time, or record audio stream.

[0061] S2, performing voice activity detection on the audio stream, and dividing the audio stream into a plurality of single-person voice segments.

[0062] For example, first, the real-time received continuous audio stream is preprocessed, the audio stream is framed according to a fixed time window, and the frames can be overlapped to maintain continuity. Each frame of signal is high-pass filtered to compensate for the high-frequency attenuation of the speech signal and enhance the subsequent feature extraction effect.

[0063] Secondly, the short-time energy of each frame of signal is calculated to reflect the intensity change of the speech signal. The number of times each frame of signal crosses the zero level is calculated to distinguish between unvoiced and voiced sounds.

[0064] Optionally, advanced acoustic features such as mel-frequency cepstral coefficients, spectral entropy, and formant frequencies can be further extracted to improve the accuracy of voice activity detection.

[0065] When the frame energy exceeds a preset first energy threshold, it is determined that the speech segment starts;

[0066] When the frame energy is lower than a preset second energy threshold, it is determined that the speech segment ends;

[0067] The first energy threshold and the second energy threshold are dynamically adjusted according to the ambient noise level. For example, in a quiet environment, the first energy threshold is lowered to avoid missing weak speech; in a noisy environment, the second energy threshold is increased to reduce false detection of non-speech signals.

[0068] From the start time of the speech segment, the speech frames are continuously recorded until the end time of the speech segment is detected, generating an independent single-person speech segment.

[0069] The short non-speech frames (such as silent gaps) at the front and back ends are removed to ensure that the segment only contains valid speech content; adjacent short speech segments are merged to avoid segment splitting due to short pauses.

[0070] As a preferred technical solution, multi-channel audio streams can also be processed in parallel to support real-time segmentation in multi-person dialogue scenarios, and a streaming computing framework that processes as it receives is used to reduce latency and ensure real-time performance.

[0071] As an optional technical solution, a lightweight neural network can be trained to input multiple frames of acoustic features and output the probability of determining "speech segment / non-speech segment" to further improve the detection accuracy in complex scenarios. The decision results of historical frames and the current frame features can also be used to optimize the stability of speech segments and non-speech segments through a state machine (such as a hidden Markov model).

[0072] S3, extract a preset number of single-person speech segments as a first sample set, and the remaining single-person speech segments as a second sample set.

[0073] The application provides two different schemes for sample division.

[0074] If the acquired audio stream is a real-time audio stream, the number of single-person voice segments is extracted in time sequence according to the acquisition of the single-person voice segments, and when the number of extracted single-person voice segments reaches a preset number, the extracted single-person voice segments are taken as a first sample set, and the subsequently continuously extracted single-person voice segments are taken as a second sample set.

[0075] If the acquired audio stream is a recorded audio stream, all acquired single-person voice segments can be randomly extracted, and a preset number of single-person voice segments are extracted as a first sample set, and the remaining unextracted single-person voice segments are taken as a second sample set.

[0076] S4, acquiring a voiceprint embedding vector corresponding to each single-person voice segment in the first sample set, performing online clustering on the voiceprint embedding vector, and generating a speaker identity pool, the speaker identity pool including a plurality of temporary identity labels and a voice model corresponding to the temporary identity labels.

[0077] It should be noted that the voice model is a deep learning model including at least a voiceprint feature vector clustering centroid, and specifically, see Figure 2 , acquiring a voiceprint embedding vector corresponding to each single-person voice segment in the first sample set, performing online clustering on the voiceprint embedding vector, and generating a speaker identity pool, including the following steps:

[0078] S401, performing voiceprint feature extraction on the segmented voice segment to generate a fixed-dimension voiceprint embedding vector.

[0079] For example, the voiceprint feature extraction method in the application can use a pre-trained neural network model to extract high-dimensional voiceprint features, and the vector represents the acoustic characteristics of the speaker; or, extract Mel frequency cepstrum coefficient, spectral centroid, formant, etc. and convert it into a fixed-length vector.

[0080] The voiceprint embedding vector is normalized to make the vectors of different length voices have comparable module length.

[0081] S402, determining the number of clustering clusters by an unsupervised clustering algorithm, and dividing the voiceprint embedding vector into the clustering clusters.

[0082] Specifically, the method for determining the number of clustering clusters includes at least one of the following methods:

[0083] The number of clustering clusters can be determined by, for example, elbow rule, silhouette coefficient, gap statistics.

[0084] The number of clustering clusters is adaptively determined by density analysis, and the number of clustering clusters can be determined by density peak clustering or density-based clustering.

[0085] The number of preset clustering clusters is dynamically incremented, and the number of clustering clusters can be determined by setting an initial clustering number and dynamically adjusting the initial clustering number according to new data.

[0086] S403、According to the centroid of the voiceprint embedding vector in the clustering cluster, a speech model is generated, and a temporary identity label is assigned to each clustering cluster.

[0087] For each clustering cluster, its centroid is calculated as the voiceprint feature representative of the cluster.

[0088] Centroid calculation formula:

[0089] Where, centroid i is the centroid vector of the i-th cluster;

[0090] n i is the number of voiceprint vectors contained in the i-th cluster;

[0091] vector j is the j-th voiceprint embedding vector in the i-th cluster.

[0092] If there are 10 voiceprint vectors (512 per dimension) in the cluster, the centroid is the dimension-wise average of the 10 vectors.

[0093] The centroid vector of each clustering cluster is taken as a speech model;

[0094] It should be noted that the model can be attached with other meta information (such as interval distribution of the speech segment corresponding to the model, emotion vector representation corresponding to the model, and text content coherence score, etc.).

[0095] A unique temporary identity label is generated for each clustering cluster for subsequent identification and tracking.

[0096] S5, calculate the multi-dimensional fusion similarity of each single-person speech segment in the second sample set and the speech model.

[0097] In a zero-configuration adaptive speaker recognition system, in order to improve the accuracy and robustness of recognition, a single voiceprint feature is often insufficient to cope with complex scenarios (such as noise, short speech, and emotional changes). Therefore, the present application can also extract multi-dimensional information (voiceprint embedding vector, time sequence continuity vector, emotion state vector, and text content coherence vector) to comprehensively represent the speech segment

[0098] Specifically, referring to Figure 3 , comprising the following steps:

[0099] S501, acquire multi-dimension information of single-person speech segment.

[0100] Specifically, the multi-dimension information includes a voiceprint embedding vector, a time sequence continuity vector, an emotion state vector, and a text content coherence vector.

[0101] Frame, pre-emphasize, and window the single-person speech segment in the second sample set for pretreatment;

[0102] Calculate a mel-frequency cepstral coefficient or a spectrum feature;

[0103] Map the spectrum feature to a high-dimensional embedding vector using a deep neural network model.

[0104] Generate a fixed-length voiceprint embedding vector for each single-person speech segment to represent the identity feature of the speaker of the current single-person speech segment.

[0105] Calculate the mean, standard deviation, and maximum interval of the intervals between the speech segments to form a time sequence continuity feature by statistically scoring the continuity of the time intervals between the current speech segment and the previous n recognized speech segments.

[0106] Thus, the same speaker in a continuous conversation can be better recognized, or the switching of different speakers can be distinguished.

[0107] Analyze the acoustic features (such as pitch, energy, and speech rate) and spectrum features of the speech using a speech emotion recognition model.

[0108] Classify the emotion into discrete categories or score the emotional intensity.

[0109] Map the emotion category or emotional intensity score to an emotion state vector, and if a change in emotion is detected (such as from calm to excited), the weight of the voiceprint similarity calculation can be adjusted.

[0110] Convert the speech segment to text using an automatic speech recognition model, extract the semantic embedding vector of the text through natural language processing technology, calculate the semantic similarity of the current text and the historical recognized text, or analyze the theme consistency, and form a text content coherence vector based on the semantic similarity, theme consistency, and other indicators.

[0111] When the voiceprint feature is disturbed by noise, the same speaker can be confirmed through the text content, or the specific language habits of different speakers can be distinguished.

[0112] S502, determine the weight of each dimension information in the multi-dimension information through a fusion decision function.

[0113] Concatenate or combine the four dimension vectors into a comprehensive feature vector;

[0114] Adjust the weight of each dimension dynamically according to the environment (such as noise level, speech length);

[0115] Specifically, the general form of the fusion decision function can be represented as:

[0116] W = f (E, S, T, C)

[0117] Where W is the dimension weight vector, E, S, T, C are the voiceprint embedding vector, time sequence continuity vector, emotional state vector, and text content coherence vector respectively; f() is the fusion function, which can be designed as linear weighting, nonlinear model (such as neural network) or rule engine according to specific requirements.

[0118] Specifically, a fixed weight template can be set according to prior knowledge. For example:

[0119] High noise scene: W = [0.4, 0.2, 0.1, 0.3], reduce voiceprint weight, increase text weight.

[0120] Short speech scene: W = [0.3, 0.4, 0.1, 0.2], to enhance time sequence continuity.

[0121] According to the recognition accuracy, false positive rate and other indicators, update the fixed weight template by gradient descent method:

[0122]

[0123] Where η is the learning rate, Loss is the recognition error function, W t is the weight of the current iteration, Y is the true value, is the predicted value.

[0124] Or, consider the weight as a hyperparameter, search for the optimal weight combination through Bayesian optimization algorithm (such as Gaussian process regression); or, construct a multi-input neural network, each dimension feature is input into an independent branch, and finally calculate the weight through a fusion layer (such as weighted sum, attention mechanism).

[0125] S503, according to the weight of each dimension information, calculate the multi-dimensional fusion similarity between single speech segment and each speech model.

[0126] Specifically, the similarity of single-dimensional information is calculated, and the similarity score is determined according to the dimension weight vector.

[0127] S6, update the speaker identity pool according to the multi-dimensional fusion similarity, and output the temporary identity label corresponding to the single speech segment.

[0128] Specifically, if the similarity score obtained by the highest similarity model is greater than or equal to the preset threshold, the information (such as a voiceprint vector, a timing feature, etc.) of the speech segment is merged into the corresponding model.

[0129] The model parameters are updated according to the information of the speech segment, for example, a weighted average voiceprint embedding vector, a cumulative timing interval statistic, an updated emotion / text feature distribution, etc.

[0130] Directly return the matched existing identity label.

[0131] If the similarity score obtained by the highest similarity model is less than the preset threshold, it is determined that it is a new speaker, a new temporary identity label is generated, the multi-dimensional information of the current speech segment is taken as the speech model parameters of the new speaker, the speech model of the new speaker is inserted at the end of the speaker identity pool, and is marked as "temporary addition state". If the same temporary label is matched continuously for multiple times in subsequent recognition, the "temporary addition state" mark is removed.

[0132] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0133] In some embodiments, the present application can further include the following steps:

[0134] A user interface is provided to allow the operator to perform at least one of the following operations:

[0135] Renaming or binding the identity of the temporary identity label;

[0136] Correcting the temporary identity label corresponding to a single speech segment;

[0137] Merging multiple temporary identity labels;

[0138] Forcing to update the speech model in the speaker identity pool to generate an optimized model.

[0139] Specifically, the present application provides a human supervision and identity management interface in parallel with the real-time recognition process. The operator (such as a conference host or a recorder) can intervene and manage the automatic recognition results of the system through the interface.

[0140] The operator can perform at least one or more of the following management operations:

[0141] Identity renaming, for example, modifying the anonymous label (such as speaker_A) automatically generated by the system to a real identity (such as Zhang San) with clear semantics.

[0142] Identity correction, e.g. when the system wrongly attributes a voice segment to speaker_A, the operator can manually correct it to the right speaker_B.

[0143] Identity merge, e.g. when the system wrongly identifies the same person as speaker_C and speaker_D, the operator can merge the two identities into one.

[0144] Each time the user supervises the system, it will be considered as a highest priority supervision signal, and will force the system to update the voice models in the "speaker identity pool", specifically:

[0145] For identity correction operation, the system will subtract the voiceprint embedding vector of the voice segment from the model of the original wrong identity (if updated), and update it to the model of the correct identity with high weight, to achieve instant correction of the model.

[0146] For identity merge operation, the system will fuse all voiceprint information under the two identities to generate a more robust and comprehensive new voice model.

[0147] Through this closed-loop optimization of human-machine cooperation, the long-term recognition accuracy and robustness of the system are continuously and rapidly improved.

[0148] Reference Figure 4 shows the complete process of a voiceprint recognition system:

[0149] The system first processes continuous audio streams, identifies valid speech segments through voice activity detection (VAD), and extracts voiceprint features to form embedding vectors.

[0150] The extracted voiceprint embedding vectors are clustered to initially form M identity models, representing different speakers.

[0151] For new voiceprint embedding vectors, the system calculates their similarity with existing identity models to determine whether to attribute to existing identities or create new identity models.

[0152] Finally, through artificial supervision, the voice models in the identity pool are managed to ensure the accuracy and reliability of the system, and closed-loop optimization is achieved.

[0153] Reference Figure 5 shows a zero-configuration adaptive speaker recognition method system, which includes:

[0154] An audio acquisition module for receiving audio streams;

[0155] A voice activity detection module for voice activity detection on the audio stream, and cutting the audio stream into multiple single-person voice segments;

[0156] The sample set division module is configured to extract a preset number of single-person voice clips as a first sample set and the remaining single-person voice clips as a second sample set.

[0157] The voiceprint embedding extraction module is configured to obtain a voiceprint embedding vector corresponding to each single-person voice clip in the first sample set.

[0158] The online clustering module is configured to perform online clustering on the voiceprint embedding vectors to generate a speaker identity pool, the speaker identity pool including a plurality of temporary identity labels and voice models corresponding to the temporary identity labels.

[0159] The multi-dimensional similarity calculation module is configured to calculate a multi-dimensional fusion similarity between each single-person voice clip in the second sample set and the voice models.

[0160] The identity pool updating module is configured to update the speaker identity pool according to the multi-dimensional fusion similarity and output a temporary identity label corresponding to the single-person voice clip.

[0161] The user interaction interface is configured to allow a user to perform at least one of the following operations:

[0162] Renaming or binding an identity to a temporary identity label;

[0163] Correcting a temporary identity label of a single-person voice clip;

[0164] Merging a plurality of temporary identity labels;

[0165] Forcing to update the voice models in the speaker identity pool to generate an optimized model.

[0166] Based on the above embodiments, it can be seen that the present application can realize efficient multi-person dialogue speaker recognition in a zero-configuration and adaptive manner, without pre-registering a voiceprint, automatically constructing an initial identity pool, and realizing rapid deployment and startup. Meanwhile, the present application adopts an incremental updating mechanism, can adapt to dynamic changes of speakers (such as joining / leaving in the middle) in real time, and fuse voiceprint, timing, emotion and text features, automatically adjust voice models and recognition thresholds, and further improve the recognition accuracy. It can also realize closed-loop optimization through a manual supervision interface, thereby continuously improving long-term recognition accuracy.

[0167] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A zero-configuration, adaptive speaker recognition method, characterized by, The method comprises the following steps: receiving an audio stream; performing voice activity detection on the audio stream to segment the audio stream into a plurality of single-person voice segments; extracting a preset number of the single-person voice segments as a first sample set, and the remaining single-person voice segments as a second sample set; obtaining a voiceprint embedding vector corresponding to each single-person voice segment in the first sample set, performing online clustering on the voiceprint embedding vector to generate a speaker identity pool, and the speaker identity pool comprising a plurality of temporary identity labels and a voice model corresponding to the temporary identity labels; calculating a multi-dimensional fusion similarity between each single-person voice segment in the second sample set and the voice model, comprising: obtaining multi-dimensional information of the single-person voice segment, the multi-dimensional information comprising a voiceprint embedding vector, a time sequence continuity vector, an emotional state vector, and a text content coherence vector; determining the weight of each dimension information in the multi-dimensional information through a fusion decision function; calculating the multi-dimensional fusion similarity between the single-person voice segment and each voice model according to the weight of each dimension information; updating the speaker identity pool according to the multi-dimensional fusion similarity, and outputting the temporary identity label corresponding to the single-person voice segment.

2. The method of claim 1, wherein, The fusion decision function adopts at least one of weighted average, Bayesian network or neural network model, and the weight is adaptively optimized according to the recognition result of the single-person voice segment.

3. The method of claim 1, wherein, The online clustering of the voiceprint embedding vector to generate the speaker identity pool comprises: determining the number of clustering clusters by an unsupervised clustering algorithm, and dividing the voiceprint embedding vector into the clustering clusters; generating a voice model according to the centroid of the voiceprint embedding vector in the clustering cluster, and assigning a temporary identity label to each clustering cluster.

4. The method of claim 3, wherein, The method for determining the number of clustering clusters comprises at least one of: determining the number of clustering clusters by a clustering effectiveness index; adaptively determining the number of clustering clusters by density analysis; dynamically incrementally correcting the number of preset clustering clusters.

5. The method of claim 1, wherein, The updating of the speaker identity pool according to the multi-dimensional fusion similarity comprises: when the multi-dimensional fusion similarity is greater than or equal to a preset threshold, determining the temporary identity label corresponding to the single-person voice segment, and updating the voice model corresponding to the temporary identity label based on the voiceprint embedding vector corresponding to the single-person voice segment at a preset learning rate; when the multi-dimensional fusion similarity is less than a preset threshold, creating a new temporary identity label and a new voice model to join the speaker identity pool.

6. The method of any one of claims 1 to 5, wherein, The method further comprises the following steps: providing a user interface to allow an operator to perform at least one of the following operations: renaming or binding the identity of the temporary identity label; correcting the temporary identity label corresponding to the single-person voice segment; merging a plurality of temporary identity labels; forcibly updating the voice model in the speaker identity pool to generate an optimized model.

7. The method of claim 6, wherein, The extraction of a preset number of single-person voice segments as a first sample set, and the remaining single-person voice segments as a second sample set, comprises: When the number of extracted single-person speech segments reaches a preset number, the extracted single-person speech segments are taken as a first sample set, and subsequent extracted single-person speech segments are taken as a second sample set. Alternatively, a preset number of single-person speech segments are randomly extracted as a first sample set, and the remaining single-person speech segments are taken as a second sample set.

8. The method of claim 7, wherein, A deep neural network model is used to obtain a voiceprint embedding vector corresponding to each single-person speech segment in the first sample set, and model parameters of the deep neural network model are dynamically adjusted in the identification process.

9. A zero-configuration adaptive speaker recognition method system, comprising: an audio acquisition module configured to receive an audio stream; a voice activity detection module configured to perform voice activity detection on the audio stream and divide the audio stream into a plurality of single-person speech segments; a sample set division module configured to extract a preset number of single-person speech segments as a first sample set and the remaining single-person speech segments as a second sample set; a voiceprint embedding extraction module configured to obtain a voiceprint embedding vector corresponding to each single-person speech segment in the first sample set; an online clustering module configured to perform online clustering on the voiceprint embedding vectors to generate a speaker identity pool, the speaker identity pool including a plurality of temporary identity labels and speech models corresponding to the temporary identity labels; a multi-dimensional similarity calculation module configured to calculate a multi-dimensional fusion similarity between each single-person speech segment in the second sample set and the speech models, including: obtaining multi-dimensional information of the single-person speech segment, the multi-dimensional information including a voiceprint embedding vector, a time sequence continuity vector, an emotion state vector, and a text content coherence vector; determining weights of each dimension of the multi-dimensional information through a fusion decision function; calculating a multi-dimensional fusion similarity between the single-person speech segment and each speech model according to the weights of the multi-dimensional information; an identity pool updating module configured to update the speaker identity pool according to the multi-dimensional fusion similarity and output a temporary identity label corresponding to the single-person speech segment; a user interaction interface configured to allow a user to perform at least one of the following operations: renaming or binding an identity of a temporary identity label; correcting a temporary identity label of a single-person speech segment; merging a plurality of temporary identity labels; forcing an update of a speech model in the speaker identity pool to generate an optimized model.

Images