A speaker segmentation method and system based on an improved variational Bayesian algorithm

Abstract

The present invention provides a speaker segmentation method based on an improved variational Bayesian algorithm, comprising: segmenting speech and extracting features of each segment; clustering using the improved variational Bayesian algorithm, first cross-calculating the feature similarity of speech segments, merging the two speech segments with the highest similarity until the maximum feature similarity is less than a threshold, and obtaining an initial clustering result; calculating the class vector of each class, and obtaining the probability of the voiceprint of each speech segment belonging to each class; calculating the voiceprint of each speech segment and the new weight of each class, and updating the initial state probability in a hidden Markov model at the same time, calculating the lower limit of evidence until a termination condition is met, and completing the classification of speech segments; merging speech segments of the same class, obtaining the start and end points of speech of different people, and realizing speaker segmentation; the method provided by the present invention combines the cosine algorithm with the improved variational Bayesian algorithm for clustering and speaker segmentation, and the segmentation is more accurate, the effect of speaker separation is improved, and it is more suitable for long dialogue scenarios.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, and in particular to a speaker segmentation method and system based on an improved variational Bayesian algorithm. Background Art

[0002] Speaker diarization is to mark the start and end points of different speakers in a piece of speech. The current popular method is to divide a piece of speech into many small segments, extract the voiceprint information and cluster them, so as to distinguish the start and end points of different speakers.

[0003] At present, voiceprints are extracted after speech segmentation and then combined with the Variational Bayes algorithm. The traditional speaker separation results are used as pre-separation input. On this basis, the features of each frame are modeled through the hidden Markov model, so that the speaker model can be successfully and robustly estimated from very short speech segments. This method can effectively shorten the size of the speaker modeling unit and achieve robust estimation of the speaker model from very short speech segments. However, since the input frame features in the original variational Bayes algorithm are simple splicing of non-silent frames, the downsampling process may merge frames that are not in the same speech segment into one state, and the final output may show that the speaker separation boundary does not match the segmentation boundary, affecting the separation effect. Summary of the invention

[0004] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a speaker segmentation method based on an improved variational Bayesian algorithm, which combines the cosine algorithm and the improved variational Bayesian algorithm for clustering and speaker segmentation. The segmentation is more accurate, the effect of speaker separation is improved, and it is more suitable for long conversation scenarios.

[0005] The present invention adopts the following technical solution:

[0006] A speaker segmentation method based on an improved variational Bayesian algorithm, comprising:

[0007] The speech is segmented and the voiceprint features of each segment are extracted to obtain the speech segment features;

[0008] The improved variational Bayesian algorithm is used for clustering. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarity are updated until the maximum feature similarity is less than the threshold, thus obtaining the initial clustering result.

[0009] According to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained;

[0010] The hidden Markov model and forward-backward algorithm are used to calculate the new weight of each voiceprint belonging to each category of each speech segment. At the same time, the initial state probability in the hidden Markov model is updated, and the lower limit of evidence is calculated to meet the termination condition. The classification of the speech segment is completed;

[0011] Merge similar speech segments, obtain the starting and ending points of different people's speech, and achieve speaker segmentation.

[0012] Specifically, the cosine algorithm is used to cross-calculate the similarity of speech segment features, specifically:

[0013]

[0014] Among them, i and j are the labels of the two speech segments, e is the speech segment feature, and a ij is the similarity between speech segment i and speech segment j.

[0015] Specifically, the updated feature similarity is:

[0016]

[0017] Among them, a' ij is the updated feature similarity, m is the number of speech segments in the category to which speech segment i belongs, and n is the number of speech segments in the category to which speech segment j belongs.

[0018] Specifically, according to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained;

[0019]

[0020]

[0021] x t =F C ||E t ||

[0022] Among them, α s represents the class vector of class s, where F A ,F B and F C is a hyperparameter, I is the identity matrix, γ ts represents the weight value of the tth segment belonging to class s, where E t is the voiceprint vector extracted from xvector, ||E t || is L2 regularization processing, Xt is the feature vector of the tth segment after scaling;

[0023]

[0024] Where D is the vector dimension extracted by xvector, φ is a column vector whose elements are all 1, and λ s For L s -1 The diagonal vector of is the scaled eigenvector x of the tth segment t The probability of occurrence in class s.

[0025] Specifically, the hidden Markov model and the forward-backward algorithm are used to calculate the new weight of each voiceprint belonging to each category, specifically:

[0026]

[0027]

[0028] where γ ts Represents the weight value of the t-th segment belonging to class s, A(t,s) is the forward algorithm based on HMM that the t-th segment is class s, B(t,s) is the backward algorithm based on HMM that the t-th segment is class s, and W is the total number of speech segments.

[0029] Specifically, the updating of the initial state probability in the hidden Markov model is as follows:

[0030]

[0031] Where s' represents the class corresponding to the t-1 segment, s represents the class corresponding to the t segment, and π s represents the starting probability of class s of hmm, γ 1s is the weight value of the first segment belonging to class s, and ∝ is proportional to the operation.

[0032] Specifically, the lower limit of evidence is calculated until the termination condition is met, wherein the calculation of the lower limit of evidence is specifically as follows:

[0033]

[0034] Where l is the lower limit of evidence, is the log sum of the diagonal elements, are the diagonal elements and .

[0035] Another aspect of the present invention provides a speaker segmentation system based on an improved variational Bayesian algorithm, comprising:

[0036] Preprocessing unit: segment the speech, extract the voiceprint features of each segment, and obtain the speech segment features;

[0037] Initial clustering unit: clustering is performed using the improved variational Bayesian algorithm. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarities are updated until the maximum feature similarity is less than the threshold, thus obtaining the initial clustering result.

[0038] Probability calculation unit: according to the initial clustering results, calculate the class vector of each class, and find the probability of each voiceprint belonging to each class;

[0039] Update unit: Use hidden Markov model and forward-backward algorithm to calculate the voiceprint of each speech segment and the new weight of each category, update the initial state probability in the hidden Markov model, calculate the lower limit of evidence to meet the termination condition, and complete the speech segment classification;

[0040] Speech segment merging unit: merges similar speech segments to obtain the starting and ending points of different people's speech and realize speaker segmentation.

[0041] Yet another embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned speaker segmentation method based on the improved variational Bayesian algorithm are implemented.

[0042] Yet another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned speaker segmentation method based on the improved variational Bayesian algorithm are implemented.

[0043] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The present invention provides a speaker segmentation method based on an improved variational Bayesian algorithm, comprising: segmenting speech, extracting voiceprint features of each segment, and obtaining speech segment features; clustering using the improved variational Bayesian algorithm, first using the cosine algorithm to cross-calculate the similarity of speech segment features, merging the two speech segments with the highest similarity, and updating the speech segment features and feature similarities until the maximum feature similarity is less than a threshold, thereby obtaining an initial clustering result; based on the initial clustering result, calculating the class vector of each class, and obtaining the voiceprint of each speech segment. The probability of occurrence of each category is obtained; the hidden Markov model and the forward-backward algorithm are used to calculate the voiceprint of each speech segment and the new weight of each category, and the initial state probability in the hidden Markov model is updated at the same time, and the lower limit of evidence is calculated to meet the termination condition, and the speech segment classification is completed; the speech segments of the same type are merged to obtain the starting and ending points of the voices of different people, and the speaker segmentation is realized; the method provided by the present invention combines the cosine algorithm with the improved variational Bayes algorithm for clustering and speaker segmentation, the segmentation is more accurate, the effect of speaker separation is improved, and it is more suitable for long dialogue scenarios.

[0045] (2) In the method provided by the present invention, the improved variational Bayes algorithm makes up for the limitation of the word order of the cosine algorithm, making the segmentation of long dialogue scenes more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a speaker segmentation method based on an improved variational Bayesian algorithm provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a topology diagram based on a hidden Markov model in an actual conversation scenario provided by an embodiment of the present invention;

[0048] Figure 3 A diagram of the architecture of a speaker segmentation system based on an improved variational Bayesian algorithm provided in an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0051] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0052] The present invention proposes a speaker segmentation method based on an improved variational Bayesian algorithm, which combines the cosine algorithm with the improved variational Bayesian algorithm for clustering and speaker segmentation. The segmentation is more accurate, the effect of speaker separation is improved, and it is more suitable for long dialogue scenarios.

[0053] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. The various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

[0054] The following is a description of a speaker segmentation method based on an improved variational Bayesian algorithm provided in an embodiment of the present application in combination with a specific application example. The execution subject of the method can be a terminal device, such as a mobile phone, a computer, a wearable device, etc., or a server, which is not limited here. Figure 1 , is a flow chart of a speaker segmentation method based on an improved variational Bayesian algorithm provided by an embodiment of the present invention, specifically comprising:

[0055] A speaker segmentation method based on an improved variational Bayesian algorithm, comprising:

[0056] S101: Segment the speech, extract the voiceprint features of each segment, and obtain the speech segment features;

[0057] For the input single-channel speech, the silence can be compressed by Voice Activity Detection (VAD), and then the compressed silenced speech can be transcoded into text by Automatic Speech Recognition (ASR), and the text alignment result can be output. At the same time, according to the text alignment result, the speech with a silence duration greater than the preset time (currently set to 0.3s) in the transcoded alignment result is further compressed to obtain multiple speech segments, the speech text corresponding to each speech segment, and the corresponding results of the speech units and the start and end times in each speech segment, also known as the alignment result of the speech segments;

[0058] The extracted speech segment feature data is used for the improved variational Bayesian model to identify the speech tags of the speech to be separated, and the feature data includes: a concatenated feature matrix and the end frame number of each speech segment.

[0059] S102: clustering is performed using an improved variational Bayesian algorithm. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarities are updated until the maximum feature similarity is less than a threshold, thereby obtaining an initial clustering result.

[0060] In a specific embodiment, the initial N speech segment features, i.e., N types of speech segments, use the cosine algorithm to cross-calculate similarities between each other to obtain an NxN similarity matrix, and merge the two most similar segments into one category, update the similarities with other categories, and obtain (N-1)x(N-1) similarities. Iterate continuously until the maximum similarity is less than the threshold.

[0061] Specifically, the cosine algorithm is used to cross-calculate the similarity of speech segment features, specifically:

[0062]

[0063] Among them, i and j are the labels of the two speech segments, e is the speech segment feature, and a ij is the similarity between speech segment i and speech segment j.

[0064] Specifically, the updated feature similarity is:

[0065]

[0066] Among them, a' ij is the updated feature similarity, m is the number of speech segments in the category to which speech segment i belongs, and n is the number of speech segments in the category to which speech segment j belongs.

[0067] S103: Calculate the class vector of each class according to the initial clustering result, and obtain the probability of each voiceprint of each speech segment belonging to each class;

[0068] Specifically, according to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained;

[0069]

[0070]

[0071] x t =F C ||E t ||

[0072] Among them, α s represents the class vector of class s, where F A ,F B and F C is a hyperparameter, γ ts represents the weight value of the tth segment belonging to class s, where E t is the voiceprint vector extracted from xvector, ||E t || is L2 regularization, Xt is the eigenvector of the tth segment after scaling, and I is the unit matrix;

[0073]

[0074] Where D is the vector dimension extracted by xvector, φ is a column vector whose elements are all 1, and λ s For L s -1 The diagonal vector of

[0075] is the scaled eigenvector x of the tth segment t The probability of occurrence on class s, where class s is speaker s, also corresponds to the state on hmm, and is therefore also the probability of occurrence of the state on hmm.

[0076] The hyperparameters in this algorithm are ahc_t, F A , F B , F C ,P loop and π need to be optimized. A , F B , P loop and π are relatively fixed, among which F A You can use 0.3, F B You can use 17, P loop You can use 0.99, π to initialize Where S represents the number of initialized speech classes. C After testing, it is found that it is related to the vector dimension D extracted from xvector, the segment duration and segment offset, and needs to be adjusted after test data before use. In the test, if a 512-dimensional voiceprint vector, 1.5s segment duration and 0.75s segment offset are used, F C You can use 75. ahc_t is the termination threshold of clustering. You need to test and select a suitable threshold, which is generally between 0.2 and 0.5.

[0077] S104: The hidden Markov model and the forward-backward algorithm are used to calculate the voiceprint of each speech segment and the new weight of each category, and the initial state probability in the hidden Markov model is updated at the same time. The lower limit of evidence is calculated until the termination condition is met, and the speech segment classification is completed;

[0078] Specifically, the hidden Markov model and the forward-backward algorithm are used to calculate the new weight of each voiceprint belonging to each category, specifically:

[0079]

[0080]

[0081] where γ ts Represents the weight value of the t-th segment belonging to class s, A(t,s) is the forward algorithm based on HMM that the t-th segment is class s, B(t,s) is the backward algorithm based on HMM that the t-th segment is class s, and W is the total number of speech segments.

[0082] like Figure 2 , a schematic diagram of a topological diagram of a hidden Markov model in an actual dialogue scenario provided by an embodiment of the present invention, wherein the figure shows a topological diagram of three speakers, each of which has a state, π is the initial state probability, P loop is the self-loop probability, δ is the occurrence probability (i.e. );

[0083] p(s|s')=(1-P loop )π s +δ(s=s')P loop

[0084] Specifically, the updating of the initial state probability in the hidden Markov model is as follows:

[0085]

[0086] Where s' represents the class corresponding to the t-1 segment, s represents the class corresponding to the t segment, and π s represents the starting probability of class s of hmm, γ 1s is the weight value of the first segment belonging to class s, and ∝ is proportional to the operation.

[0087] Therefore, after this step is completed, Perform normalization.

[0088] Specifically, the lower limit of evidence is calculated until the termination condition is met, wherein the calculation of the lower limit of evidence is specifically as follows:

[0089]

[0090] Where l is the lower limit of evidence, is the log sum of the diagonal elements, The termination condition is generally when l converges or reaches the maximum number of iterations. The convergence is determined when the difference between two consecutive l values ​​is less than a predetermined value, which is generally 10. -6 .

[0091] S105: Merge similar speech segments to obtain the start and end points of different people's speech to achieve speaker segmentation.

[0092] Each speech segment is classified, and the speech segments of the same classification are merged and combined into several large segments. The starting time and ending time of these speech segments are the starting and ending time points of this type of speech.

[0093] During the specific verification, the telephone recordings were segmented and clustered. Since the voiceprint model used adopted the cosine method for similarity, the clustering-based hierarchical clustering algorithm was directly used for segmentation and clustering, and the obtained index was der=2.9%, where der is an index for evaluating the effect of speaker segmentation and clustering. The smaller the value, the better the effect. For the same telephone recording, the method proposed by the present invention can be used to optimize der=2.2%, which has a significant improvement effect.

[0094] like Figure 3 On the other hand, an embodiment of the present invention provides a speaker segmentation system based on an improved variational Bayesian algorithm, comprising:

[0095] Preprocessing unit 301: segment the speech, extract the voiceprint features of each segment, and obtain the speech segment features;

[0096] For the input single-channel speech, the silence can be compressed by Voice Activity Detection (VAD), and then the compressed silenced speech can be transcoded into text by Automatic Speech Recognition (ASR), and the text alignment result can be output. At the same time, according to the text alignment result, the speech with a silence duration greater than the preset time (currently set to 0.3s) in the transcoded alignment result is further compressed to obtain multiple speech segments, the speech text corresponding to each speech segment, and the corresponding results of the speech units and the start and end times in each speech segment, also known as the alignment result of the speech segments;

[0097] The extracted speech segment feature data is used for the improved variational Bayesian model to identify the speech tags of the speech to be separated, and the feature data includes: a concatenated feature matrix and the end frame number of each speech segment.

[0098] Initial clustering unit 302: clustering is performed using an improved variational Bayesian algorithm. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarities are updated until the maximum feature similarity is less than a threshold, thereby obtaining an initial clustering result.

[0099] In a specific embodiment, the initial N speech segment features, i.e., N types of speech segments, use the cosine algorithm to cross-calculate similarities between each other to obtain an NxN similarity matrix, and merge the two most similar segments into one category, update the similarities with other categories, and obtain (N-1)x(N-1) similarities. Iterate continuously until the maximum similarity is less than the threshold.

[0100] Specifically, the cosine algorithm is used to cross-calculate the similarity of speech segment features, specifically:

[0101]

[0102] Among them, i and j are the labels of the two speech segments, e is the speech segment feature, and a ij is the similarity between speech segment i and speech segment j.

[0103] Specifically, the updated feature similarity is:

[0104]

[0105] Among them, a' ij is the updated feature similarity, m is the number of speech segments in the category to which speech segment i belongs, and n is the number of speech segments in the category to which speech segment j belongs.

[0106] Probability calculation unit 303: calculates the class vector of each class according to the initial clustering result, and obtains the probability of each voiceprint belonging to each class;

[0107] Specifically, according to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained;

[0108]

[0109]

[0110] x t =F C ||E t ||

[0111] Among them, α s represents the class vector of class s, where F A ,F B and F C is a hyperparameter, γ ts represents the weight value of the tth segment belonging to class s, where E t is the voiceprint vector extracted from xvector, ||E t || is L2 regularization, Xt is the eigenvector of the tth segment after scaling, and I is the unit matrix;

[0112]

[0113] Where D is the vector dimension extracted by xvector, φ is a column vector whose elements are all 1, and λ s For L s -1 The diagonal vector of is the scaled eigenvector x of the tth segment t The probability of occurrence on class s, where class s is speaker s, also corresponds to the state on hmm, and is therefore also the probability of occurrence of the state on hmm.

[0114] The hyperparameters in this algorithm are ahc_t, F A , F B , F C ,P loop and π need to be optimized. A , F B P loop and π are relatively fixed, among which F A You can use 0.3, F B You can use 17, P loop You can use 0.99, π to initialize Where S represents the number of initialized speech classes.C After testing, it is found that it is related to the vector dimension D extracted from xvector, the segment duration and segment offset, and needs to be adjusted after test data before use. In the test, if a 512-dimensional voiceprint vector, 1.5s segment duration and 0.75s segment offset are used, F C You can use 75. ahc_t is the termination threshold of clustering. You need to test and select a suitable threshold, which is generally between 0.2 and 0.5.

[0115] Update unit 304: uses hidden Markov model and forward-backward algorithm to calculate the new weight of each voiceprint of each speech segment belonging to each category, and updates the initial state probability in the hidden Markov model, calculates the lower limit of evidence until the termination condition is met, and the speech segment classification is completed;

[0116] Specifically, the hidden Markov model and the forward-backward algorithm are used to calculate the new weight of each voiceprint belonging to each category, specifically:

[0117]

[0118]

[0119] where γ ts Represents the weight value of the t-th segment belonging to class s, A(t,s) is the forward algorithm based on HMM that the t-th segment is class s, B(t,s) is the backward algorithm based on HMM that the t-th segment is class s, and W is the total number of speech segments.

[0120] like Figure 2 , a schematic diagram of a topological diagram of a hidden Markov model in an actual dialogue scenario provided by an embodiment of the present invention, wherein the figure shows a topological diagram of three speakers, each of which has a state, π is the initial state probability, P loop is the self-loop probability, δ is the occurrence probability (i.e. );

[0121] p(s|s')=(1-P loop )π s +δ(s=s')P loop

[0122] Specifically, the updating of the initial state probability in the hidden Markov model is as follows:

[0123]

[0124] Where s' represents the class corresponding to the t-1 segment, s represents the class corresponding to the t segment, and π s represents the starting probability of class s of hmm, γ 1s is the weight value of the first segment belonging to class s, and ∝ is proportional to the operation.

[0125] Therefore, after this step is completed, Perform normalization.

[0126] Specifically, the lower limit of evidence is calculated until the termination condition is met, wherein the calculation of the lower limit of evidence is specifically as follows:

[0127]

[0128] Where l is the lower limit of evidence, is the log sum of the diagonal elements, The termination condition is generally when l converges or reaches the maximum number of iterations. The convergence is determined when the difference between two consecutive l values ​​is less than a predetermined value, which is generally 10. -6 .

[0129] Speech segment merging unit 305: merges speech segments of the same type, obtains the start and end points of speech of different people, and realizes speaker segmentation.

[0130] Each speech segment is classified, and the speech segments of the same classification are merged and combined into several large segments. The starting time and ending time of these speech segments are the starting and ending time points of this type of speech.

[0131] Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 420 and executable on the processor 420. When the processor 420 executes the computer program 411, a speaker segmentation method based on an improved variational Bayesian algorithm provided by an embodiment of the present invention is implemented.

[0132] Since the electronic device introduced in this embodiment is the device used to implement the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, the technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present invention falls within the scope of protection of the present invention.

[0133] See also Figure 5 , Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0134] like Figure 5As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, a speaker segmentation method based on an improved variational Bayesian algorithm provided by an embodiment of the present invention is implemented;

[0135] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] The present invention provides a speaker segmentation method based on an improved variational Bayesian algorithm, comprising: segmenting speech, extracting voiceprint features of each segment, and obtaining speech segment features; clustering using the improved variational Bayesian algorithm, firstly using the cosine algorithm to cross-calculate the similarity of speech segment features, merging the two speech segments with the highest similarity, and updating the speech segment features and feature similarities until the maximum feature similarity is less than a threshold value, thereby obtaining an initial clustering result; according to the initial clustering result, calculating the class vector of each class, and obtaining the voiceprint attribute of each speech segment; The method adopts a hidden Markov model and a forward-backward algorithm to calculate the voiceprint of each speech segment and the new weight of each category, and updates the initial state probability in the hidden Markov model at the same time, calculates the lower limit of evidence to meet the termination condition, and completes the classification of the speech segments; merges the speech segments of the same type, obtains the starting and ending points of the voices of different people, and realizes speaker segmentation; the method provided by the present invention combines the cosine algorithm with the improved variational Bayes algorithm for clustering and speaker segmentation, and the segmentation is more accurate, the effect of speaker separation is improved, and it is more suitable for long dialogue scenarios.

[0138] In the method provided by the present invention, the improved variational Bayes algorithm makes up for the limitation of the word order of the cosine algorithm, making the segmentation of long dialogue scenes more accurate.

[0139] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A speaker segmentation method based on an improved variational Bayesian algorithm, characterized in that: include: The speech is segmented and the voiceprint features of each segment are extracted to obtain the speech segment features; The improved variational Bayesian algorithm is used for clustering. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarity are updated until the maximum feature similarity is less than the threshold, thus obtaining the initial clustering result. According to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained, including: x t =F C ||E t ||; Among them, α s represents the class vector of class s, where F A ,F B and F C is a hyperparameter, I is the identity matrix, γ ts represents the weight value of the tth segment belonging to class s, where E t is the voiceprint vector extracted from xvector, ||E t || is L2 regularization, x t is the feature vector of the tth segment after scaling; Where D is the vector dimension extracted by xvector, φ is a column vector whose elements are all 1, and λ s For L s -1 The diagonal vector of is the scaled eigenvector x of the tth segment t Probability of occurrence in class s; The hidden Markov model and forward-backward algorithm are used to calculate the new weight of each voiceprint belonging to each category, specifically: where γ ts represents the weight value of the t-th segment belonging to class s, A(t,s) is the forward algorithm based on HMM that the t-th segment belongs to class s, B(t,s) is the backward algorithm based on HMM that the t-th segment belongs to class s, W is the total number of speech segments, and the initial state probability in the hidden Markov model is updated at the same time, and the lower limit of evidence is calculated to meet the termination condition, and the speech segment classification is completed; Merge similar speech segments, obtain the starting and ending points of different people's speech, and achieve speaker segmentation.

2. The speaker segmentation method based on the improved variational Bayesian algorithm according to claim 1, characterized in that: The cosine algorithm is used to calculate the similarity of speech segment features in pairs, specifically: Among them, i and j are the labels of the two speech segments, e is the speech segment feature, and a ij is the similarity between speech segment i and speech segment j.

3. The speaker segmentation method based on the improved variational Bayesian algorithm according to claim 2, characterized in that: The update feature similarity is specifically: Among them, a' ij is the updated feature similarity, m is the number of speech segments in the category to which speech segment i belongs, and n is the number of speech segments in the category to which speech segment j belongs.

4. The speaker segmentation method based on the improved variational Bayesian algorithm according to claim 1, characterized in that: The updating of the initial state probability in the hidden Markov model is specifically: Where s' represents the class corresponding to the t-1 segment, s represents the class corresponding to the t segment, and π s represents the starting probability of class s of hmm, γ 1s is the weight value of the first segment belonging to class s, and ∝ is proportional to the operation.

5. The speaker segmentation method based on the improved variational Bayesian algorithm according to claim 4, characterized in that: Calculate the lower limit of evidence until the termination condition is met, wherein the calculation of the lower limit of evidence is specifically as follows: Where l is the lower limit of evidence, is the log sum of the diagonal elements, are the diagonal elements and .

6. A speaker segmentation system based on an improved variational Bayesian algorithm, characterized in that: include: Preprocessing unit: segment the speech, extract the voiceprint features of each segment, and obtain the speech segment features; Initial clustering unit: clustering is performed using the improved variational Bayesian algorithm. First, the cosine algorithm is used to cross-calculate the similarity of speech segment features, the two speech segments with the highest similarity are merged, and the speech segment features and feature similarities are updated until the maximum feature similarity is less than the threshold, thus obtaining the initial clustering result. Probability calculation unit: According to the initial clustering results, the class vector of each class is calculated, and the probability of each voiceprint belonging to each class is obtained, including: x t =F C ||E t ||; Among them, α s represents the class vector of class s, where F A ,F B and F C is a hyperparameter, I is the identity matrix, γ ts represents the weight value of the tth segment belonging to class s, where E t is the voiceprint vector extracted from xvector, ||E t || is L2 regularization, x t is the feature vector of the tth segment after scaling; Where D is the vector dimension extracted by xvector, φ is a column vector whose elements are all 1, and λ s For L s -1 The diagonal vector of is the scaled eigenvector x of the tth segment t Probability of occurrence in class s; Update unit: Use hidden Markov model and forward-backward algorithm to calculate the new weight of each voiceprint belonging to each category, specifically: where γ ts represents the weight value of the t-th segment belonging to class s, A(t,s) is the forward algorithm based on HMM that the t-th segment belongs to class s, B(t,s) is the backward algorithm based on HMM that the t-th segment belongs to class s, W is the total number of speech segments, and the initial state probability in the hidden Markov model is updated at the same time, and the lower limit of evidence is calculated to meet the termination condition, and the speech segment classification is completed; Speech segment merging unit: merges similar speech segments to obtain the starting and ending points of different people's speech and realize speaker segmentation.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the method steps described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images