A flexible off-, online voiceprint logging system

Abstract

The application discloses a flexible offline and online voiceprint log system, which comprises an online branch or an online branch and an offline branch, and the output of the online branch comprises time boundaries corresponding to each speaker and voiceprints corresponding to each speaker, and the speaker boundaries can be obtained accurately to the frame level through the offline branch. The flexible offline and online voiceprint log system provided by the application meets the requirements of real-time display or high accuracy of voiceprint logs in different use scenarios, innovatively uses an EEND model to detect speaker overlap in real time, avoids extracting the voiceprints of overlapping speech for clustering, innovatively uses an Adaptive PLDA model to improve the intra-class aggregation degree and inter-class distinguishability of the voiceprints, continuously corrects the PLDA model parameters in the use process of the voiceprint system, and makes the voiceprint model more accurate with use, and innovatively uses a target speaker VAD (TS-VAD) to detect the time boundaries of different speakers in speech.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a flexible offline and online voiceprint logging system. Background Technology

[0002] Voiceprint logging technology is an important branch of speech recognition. It solves the problem of identifying who is speaking when in a dialogue scenario, and is therefore widely used in various scenarios such as meetings, classroom teaching, judicial recording, and law enforcement evidence collection. Voiceprint logging essentially involves extracting the speaker's voiceprint from a dialogue using a voiceprint model, then segmenting and clustering the voiceprints to determine the speaker at each moment. Depending on the usage, voiceprint logging can be divided into online and offline methods. The difference lies in the method itself. Online methods display the current speaker in real time through online clustering, which can be deployed on terminal devices or in scenarios with high real-time requirements. However, the clustering accuracy is lower. The main reason for the lower accuracy of online clustering is that it cannot detect speech overlap in the audio. Extracting voiceprints from overlapping speech will cause the clustering algorithm to generate a large number of clusters that cannot be matched with the speaker. In addition, the clustering algorithm relies on the distance metric between voiceprints. If the distance metric cannot guarantee a sufficiently large inter-class distance and a sufficiently small intra-class distance, online clustering is also prone to misclassification. Offline voiceprint logging requires acquiring all dialogue segments and then using spectral clustering or Bayesian Hidden Markov Clustering. Offline clustering can take into account the temporal sequence or topological relationships of different speakers in the distance matrix of the dialogue. Although the accuracy is high, the computational complexity is high, making it difficult to deploy on terminal devices. Furthermore, traditional offline voiceprint logging systems cannot handle overlapping parts of speech. Therefore, for use cases with high overlap rates, the accuracy drops sharply. Summary of the Invention

[0003] To address the technical problems existing in the prior art, the purpose of this invention is to provide a flexible offline and online voiceprint logging system.

[0004] To achieve the above objectives and technical effects, the technical solution adopted by this invention is as follows: A flexible online and offline voiceprint logging system includes an online branch, or an online branch and an offline branch. The output of the online branch includes the time boundary and the voiceprint of each speaker. The offline branch can obtain speaker boundaries accurate to the frame level.

[0005] In a flexible offline / online voiceprint logging system provided by this invention, the processing steps for the online branch include: S1. Obtain the audio sequence; S2. Window the audio sequence; S3. Use the speaker probability output by EEND to determine whether there is a speaker and whether there is speaker overlap in each frame; S4. Based on the speaker probability of each frame in step S3, cut out the silent segment and cut out the overlapping and non-overlapping segments; for speech segments that do not contain overlap, extract their voiceprints and cluster them; for overlapping speech, separate the voiceprints corresponding to the speakers in the overlapping parts, and then cluster the separated voiceprints to obtain the speakers corresponding to the overlapping segments.

[0006] In the flexible online and offline voiceprint logging system provided by this invention, the steps of extracting voiceprints and clustering non-overlapping speech segments include: S9. Extract the voiceprint of non-overlapping segments. The voiceprint model consists of ResNet and statistical pooling layers. ResNet is responsible for abstracting the frame-level speaker representation of non-overlapping segments. Then, the statistical pooling layers are used to calculate the mean μ and variance σ of all frame-level representations. The two are then concatenated to form the voiceprint of the speech segment. in, This indicates that the mean μ and variance σ are concatenated. S10. Whiten the voiceprint ZCA whitening is performed on voiceprint E to reduce redundancy in step S9: Wherein, the covariance matrix of E eigenvalues The eigenvector is U; S11. Calculate the log-likelihood ratio between the voiceprint and the cluster center; S12. Clustering based on log-likelihood ratio.

[0007] In the flexible online and offline voiceprint logging system provided by this invention, step S11 uses K-means clustering, where each cluster represents a speaker, and the cluster center is assumed to be... voiceprint With cluster center The formula for calculating the log-likelihood ratio is: ; in, , It is the projection matrix of PLDA. The covariance of the PLDA model, The number of voiceprints in the cluster.

[0008] In the flexible offline and online voiceprint logging system provided by this invention, when the voiceprint... With cluster center When the distance is greater than the threshold, Join the cluster and update the cluster center. Otherwise, the purity of human-generated voiceprints is insufficient, and they will not be added to the cluster to avoid data contamination.

[0009] In the flexible online and offline voiceprint logging system provided by this invention, when the training and testing domains do not match, the generated voiceprints are saved and continuously adapted to the PLDA model during online clustering to make it closer to the voiceprint distribution of the usage scenario, thereby making the voiceprint log more and more accurate over use. The system includes the following steps: S14: Input the initial PLDA model; S15: Update PLDA parameters using the Adaptive PLDA algorithm; Once a certain number of voiceprints are collected... p Then, the Adaptive PLDA algorithm is used to update the inter-class and intra-class covariance of PLDA, making the parameters of PLDA more suitable for the application scenario.

[0010] In the flexible online and offline voiceprint logging system provided by this invention, for overlapping speech, the steps of separating the voiceprints corresponding to the speakers in the overlapping parts, and then clustering the separated voiceprints to obtain the speakers corresponding to the overlapping segments include: S5. Extract speaker representation x for each frame using a voiceprint model. t ; S6. Calculate the voiceprints E of all speakers in overlapping speech. i ; S7, E i ZCA whitening was performed, and then the log-likelihood ratio between the whitened voiceprint and the cluster center was calculated. S8. Clustering: Subsequently, the clustering results of overlapping and non-overlapping speech are integrated to obtain the time annotations of the speakers in the complete audio.

[0011] In the flexible offline and online voiceprint logging system provided by this invention, in step S6, the speaker proportion of each frame is obtained from the EEND model in step S2. Separate the first overlapping speech. Average value per speaker With variance : Where M is the speaker representation x t The number of frames; Thus, the voiceprint of the i-th speaker in the overlapping speech is obtained. .

[0012] In a flexible offline and online voiceprint logging system provided by this invention, the processing steps for the offline branch include: S21. Obtain the voiceprint from the online voiceprint log; S22. Obtain the audio sequence; S23. Extract the Fbank features of the audio sequence; S24. The Fbank features from step S23 are concatenated with the voiceprints of each speaker in the online voiceprint log from step S21 and used as the input to the TS-VAD model. S25: Obtain the results of offline clustering.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1) This invention discloses a flexible online and offline voiceprint logging system to meet the needs of voiceprint logs for real-time display or high accuracy in different usage scenarios. If the results of the voiceprint log need to be displayed in real time, only the online branch of this invention needs to be performed. If the user needs more refined and accurate annotations, the offline branch can be used to perform fine segmentation and clustering based on the online branch. 2) This invention solves the problem of abnormal clusters caused by speech overlap in online voiceprint logs. It innovatively uses the EEND model to detect speaker overlap in speech in real time, avoiding the need to extract the voiceprints of overlapping speech for clustering. At the same time, EEND can not only provide speech overlap but also the VAD of the speech, eliminating the need for the VAD model in online branching. 3) This invention solves the problem of discriminative power in voiceprint distance measurement. It innovatively employs an Adaptive PLDA model to improve the intra-class clustering and inter-class discriminative power of voiceprints. The PLDA model parameters are continuously corrected during the use of the voiceprint system, making the voiceprint model increasingly accurate with each use. During online branch clustering, clusters are continuously adjusted to ensure that the clustering results do not produce anomalies. 4) This invention addresses the problem of speech overlap that cannot be resolved by offline voiceprint logs based on voiceprint clustering. It innovatively uses Target Speaker Voiceover Analysis (TS-VAD) to detect the temporal boundaries of different speakers in the speech. Specifically, the voiceprints corresponding to the speakers obtained in the online branch are fed into the TS-VAD model in the offline branch. TS-VAD can provide the temporal boundaries of the corresponding speakers. Because the online branch of this invention eliminates the influence of overlapping speech on speaker voiceprints, the speaker boundaries provided by TS-VAD are more accurate. Attached Figure Description

[0014] Figure 1 This is a flowchart of the present invention; Figure 2 This is a flowchart of the online branching process of the present invention; Figure 3 This is a flowchart of the offline branch of the present invention. Detailed Implementation

[0015] The present invention will now be described in detail so that its advantages and features can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0016] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0017] like Figure 1-3 As shown, a flexible online and offline voiceprint logging system includes online branches and offline branches. The output of the online branch includes: (1) the time boundary corresponding to each speaker; and (2) the voiceprint corresponding to each speaker. After the online branch ends, the user can select the offline branch to obtain speaker boundaries accurate to the frame level. The offline branch needs to use the voiceprints of each speaker in the online branch.

[0018] like Figure 2 As shown, the steps for online branching include: S1. Obtain the audio sequence; S2. Window the audio sequence; S3. Use the EEND model to obtain the probability of each speaker in each frame; The EEND model assumes that there are at most N speakers in each segment extracted in step S2. The model outputs N nodes, which represent the probability of each speaker in each frame. Based on the probability distribution, it can be determined whether there are speakers in each frame and whether there is speaker overlap. S4. Based on the speaker probability of each frame in step S3, cut out the silent segment and cut out the overlapping and non-overlapping segments. If all N nodes output by the EEND model are less than the threshold, the segment is considered a silent segment. For silent segments, there is no need to segment and cluster them; they can be directly cut off. If only one of the N nodes is greater than the threshold, the segment is considered to be non-overlapping. For speech segments that are non-overlapping, we can directly extract the voiceprint and cluster them, see steps S9~S12. If more than one of the N nodes exceeds the threshold, it is considered that there is speaker overlap in the segment. For overlapping speech, we can assume that the voiceprint of the segment is composed of overlapping speakers. Based on the overlap ratio, the voiceprints corresponding to the overlapping parts can be separated. Then, the separated voiceprints are clustered to determine the speaker corresponding to the overlapping segment, as shown in steps S5~S8: S5. Calculate frame-level speaker representation This step uses the same speaker model as in step S9 to extract the speaker representation x for each frame. t ; S6. Calculate the voiceprints of all speakers in overlapping speech. Based on the speaker weight obtained from the EEND model in step S2, the percentage of speakers in each frame is calculated. t,i We can separate the first word in the overlapping speech. Average value per speaker With variance : Where M is the speaker representation x t The number of frames; Therefore, we can obtain the voiceprint of the i-th speaker in the overlapping speech. ; S7, E i ZCA whitening was performed, and then the log-likelihood ratio between the whitened voiceprint and the cluster center was calculated. S8. Clustering: Subsequently, the clustering results of overlapping and non-overlapping speech are integrated to obtain the time annotations of the speakers in the complete audio.

[0019] The steps for extracting and clustering voiceprints include: S9. Extract the voiceprint of non-overlapping segments. The voiceprint model consists of ResNet and statistical pooling layers. ResNet is responsible for abstracting the frame-level speaker representation of non-overlapping segments. Then, the statistical pooling layers are used to calculate the mean μ and variance σ of all frame-level representations. The two are then concatenated to form the voiceprint of the speech segment. in, This indicates that the mean μ and variance σ are concatenated. S10. Whiten the voiceprint ZCA whitening is performed on voiceprint E to reduce redundancy in step S9: in covariance matrix eigenvalues , eigenvectors are ; S11. Calculate the log-likelihood ratio between the voiceprint and the cluster center. The clustering method used in this invention is K-means, where each cluster represents a speaker. To determine the speaker to whom a voiceprint belongs, it is necessary to calculate a whitened voiceprint. The distance to the cluster center was calculated using the apparent likelihood ratio. Specifically, assuming the cluster center is... voiceprint With cluster center The formula for calculating the log-likelihood ratio is: ; in, , It is the projection matrix of PLDA. The covariance of the PLDA model, The number of voiceprints in the cluster; S12. Clustering based on log-likelihood ratio Similar to K-means, based on voiceprint u p Clustering is performed based on the distance to the cluster center (log-likelihood ratio) to determine the speaker to which each audio segment belongs. To make the clustering more stable, this invention continuously merges similar clusters during the clustering process. Specifically, the log-likelihood ratio between the centers of each cluster is calculated. If the likelihood ratio exceeds a threshold, it indicates that the two clusters belong to the same speaker, and the two are merged.

[0020] In addition, when voiceprint With cluster center When the distance is greater than the threshold, Join the cluster and update the cluster center. Otherwise, the purity of human-generated voiceprints is insufficient, and they will not be added to the cluster to avoid data contamination.

[0021] Given the complex and varied environments in which voiceprint models are used, the differences in channel transfer functions on different deployment devices, the inability of training data for voiceprint models to cover all usage scenarios, and the unavoidable mismatch between training and testing domains, this can lead to a sharp decline in the performance of voiceprint models in certain scenarios.

[0022] To address the aforementioned domain mismatch issue, this invention performs online clustering while saving the generated voiceprints and continuously adapting the PLDA model to make it closer to the voiceprint distribution of the usage scenario, thereby making the voiceprint logs more and more accurate over time. The specific steps are as follows: S14: Input the initial PLDA model S15: Update PLDA parameters using the Adaptive PLDA algorithm. Specifically, once a certain number of voiceprints are collected... Subsequently, the Adaptive PLDA algorithm was used to update the inter-class and intra-class covariance of PLDA, making the parameters of PLDA more suitable for the application scenario.

[0023] The steps for offline branching include: S21. Obtain the voiceprint from the online voiceprint log. Specifically, the cluster centers of all clusters in the online voiceprint logs are used as the voiceprints of each speaker. The TS-VAD model combines the Fbank features of the audio with the corresponding speaker's voiceprint to output the speaker boundary for that speaker. S22. Obtain the audio sequence; S23. Extract the Fbank features of the audio sequence; S24. The Fbank features from step S23 are concatenated with the voiceprints of each speaker in the online voiceprint log from step S21 and used as the input to the TS-VAD model. The essence of the target speaker's VAD is a VAD model that combines voiceprints. It can provide the VAD corresponding to the speaker based on the voiceprint. In this invention, ResNet is used as the backbone network for TS-VAD. Since this VAD is frame-level, the accuracy of the offline branch is higher. S25: Obtain the results of offline clustering.

[0024] Example 1

[0025] like Figure 1-3 As shown, a flexible online and offline voiceprint logging system includes online branches and offline branches. The output of the online branch includes: (1) the time boundary corresponding to each speaker; and (2) the voiceprint corresponding to each speaker. After the online branch ends, the user can select the offline branch to obtain speaker boundaries accurate to the frame level. The offline branch needs to use the voiceprints of each speaker in the online branch.

[0026] like Figure 2 As shown, the steps for online branching include: S1. Obtain the audio sequence; S2. Window the audio sequence, dividing it into segments with a window length of 5 seconds and a window shift of 0.5 seconds; S3. Construct the EEND model The EEND model assumes that there are at most 4 speakers in each segment extracted in step S2. The model outputs 4 nodes, which represent the probability of each speaker in each frame. Based on the probability distribution, it can be determined whether there are speakers in each frame and whether there is speaker overlap. S4. Based on the speaker probabilities of each frame in step S3, cut out the silent segments and extract the overlapping and non-overlapping segments. If all four nodes output by the EEND model are less than the threshold, the segment is considered a silent segment. For silent segments, there is no need to segment and cluster them; they can be directly cut off. If only one of the four nodes is greater than the threshold, the segment is considered to be non-overlapping. For speech segments that are non-overlapping, we can directly extract the voiceprint and cluster them, see steps S9~S12. If more than one of the four nodes exceeds the threshold, it is considered that there is speaker overlap in the segment. For overlapping speech, we can assume that the voiceprint of the segment is composed of overlapping speakers. Based on the overlap ratio, the voiceprints corresponding to the overlapping parts of the speech can be separated. Then, the separated voiceprints are clustered to determine the speaker corresponding to the overlapping segment, as shown in steps S5~S8: S5. Calculate frame-level speaker representation This step uses the same speaker model as in step S9 to extract the speaker representation x for each frame. t ; S6. Calculate the voiceprints of all speakers in overlapping speech. Based on the speaker proportions obtained from the EEND model in step S2 We can separate the first word in the overlapping speech. Average value per speaker With variance : Where M is the speaker representation x t The number of frames.

[0027] Therefore, we can obtain the voiceprint of the i-th speaker in the overlapping speech. The speaker's voiceprint ; S7, E i ZCA whitening was performed, and then the log-likelihood ratio between the whitened voiceprint and the cluster center was calculated. S8. Clustering: Subsequently, the clustering results of overlapping and non-overlapping speech are integrated to obtain the time annotations of the speakers in the complete audio.

[0028] The steps for extracting and clustering voiceprints include: S9. Extract the voiceprint of non-overlapping segments. The voiceprint model consists of ResNet and statistical pooling layers. ResNet is responsible for abstracting the frame-level speaker representations of non-overlapping segments, and then the statistical pooling layers are used to calculate the mean of all frame-level representations. With variance The two are then combined to form the voiceprint of that speech segment: in, This indicates that the mean μ and variance σ are concatenated. S10. Whiten the voiceprint ZCA whitening is performed on voiceprint E to reduce redundancy in step S9: in covariance matrix eigenvalues , eigenvectors are ; S11. Calculate the log-likelihood ratio between the voiceprint and the cluster center. The clustering method used in this invention is K-means, where each cluster represents a speaker. To determine the speaker to whom a voiceprint belongs, it is necessary to calculate a whitened voiceprint. The distance to the cluster center was calculated using the apparent likelihood ratio. Specifically, assuming the cluster center is... The log-likelihood ratio is calculated as follows: ; in, , It is the projection matrix of PLDA. The covariance of the PLDA model, The number of voiceprints in the cluster.

[0029] S12. Clustering based on log-likelihood ratio Similar to K-means, based on voiceprint u p Clustering is performed based on the distance to the cluster center (log-likelihood ratio) to determine the speaker to which each audio segment belongs. To make the clustering more stable, this invention continuously merges similar clusters during the clustering process. Specifically, the log-likelihood ratio between the centers of each cluster is calculated. If the likelihood ratio exceeds a threshold, it indicates that the two clusters belong to the same speaker, and the two are merged.

[0030] In addition, when voiceprint With cluster center When the distance is greater than the threshold, Join the cluster and update the cluster center. Otherwise, the purity of human-generated voiceprints is insufficient, and they will not be added to the cluster to avoid data contamination.

[0031] Given the complex and varied environments in which voiceprint models are used, the differences in channel transfer functions on different deployment devices, the inability of training data for voiceprint models to cover all usage scenarios, and the unavoidable mismatch between training and testing domains, this can lead to a sharp decline in the performance of voiceprint models in certain scenarios.

[0032] To address the aforementioned domain mismatch issue, this invention performs online clustering while saving the generated voiceprints and continuously adapting the PLDA model to make it closer to the voiceprint distribution of the usage scenario, thereby making the voiceprint logs more and more accurate over time. The specific steps are as follows: S14: Input the initial PLDA model S15: Update PLDA parameters using the Adaptive PLDA algorithm. Specifically, once a certain number of voiceprints are collected... p Subsequently, the Adaptive PLDA algorithm was used to update the inter-class and intra-class covariance of PLDA, making the parameters of PLDA more suitable for the application scenario.

[0033] The steps for offline branching include: S21. Obtain the voiceprint from the online voiceprint log. Specifically, the cluster centers of all clusters in the online voiceprint logs are used as the voiceprints of each speaker. The TS-VAD model combines the Fbank features of the audio with the corresponding speaker's voiceprint to output the speaker boundary for that speaker. S22. Obtain the audio sequence; S23. Extract the Fbank features of the audio sequence; S24. The Fbank features from step S23 are concatenated with the voiceprints of each speaker in the online voiceprint log from step S21 and used as the input to the TS-VAD model. The essence of the target speaker's VAD is a VAD model that combines voiceprints. It can provide the VAD corresponding to the speaker based on the voiceprint. In this invention, ResNet is used as the backbone network for TS-VAD. Since this VAD is frame-level, the accuracy of the offline branch is higher. S25: Obtain the results of offline clustering.

[0034] Any parts or structures not specifically described in this invention can be made using existing technologies or products, and will not be elaborated upon here.

[0035] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A flexible offline and online voiceprint logging system, characterized in that, It includes online branches and offline branches. The output of the online branch includes the time boundary and the voiceprint of each speaker. The offline branch can obtain speaker boundaries accurate to the frame level. The processing steps for the online branch include: S1. Obtain the audio sequence; S2. Window the audio sequence; S3. Use the speaker probability output by EEND to determine whether there is a speaker and whether there is speaker overlap in each frame; S4. Based on the speaker probability of each frame in step S3, cut out the silent segment and cut out the overlapping and non-overlapping segments; for speech segments that do not contain overlap, extract their voiceprints and cluster them; for overlapping speech, separate the voiceprints corresponding to the speakers in the overlapping part, and then cluster the separated voiceprints to obtain the speakers corresponding to the overlapping segments. For speech segments that do not overlap, the steps for extracting speakerprints and clustering them include: S9. Extract the voiceprint of non-overlapping segments. The voiceprint model consists of ResNet and statistical pooling layers. ResNet is responsible for abstracting the frame-level speaker representation of non-overlapping segments. Then, the statistical pooling layers are used to calculate the mean μ and variance σ of all frame-level representations. The two are then concatenated to form the voiceprint of the speech segment. ； in, This indicates that the mean μ and variance σ are concatenated. S10. Whiten the voiceprint ZCA whitening is performed on voiceprint E to reduce redundancy in step S9: ； Wherein, the covariance matrix of E eigenvalues The eigenvector is U; S11. Calculate the log-likelihood ratio between the voiceprint and the cluster center; S12. Clustering based on log-likelihood ratio; When the training and testing domains do not match, the generated voiceprints are saved during online clustering and the PLDA model is continuously adapted to make it closer to the voiceprint distribution of the use case, thereby making the voiceprint logs more and more accurate over time. This includes the following steps: S14: Input the initial PLDA model; S15: Update PLDA parameters using the Adaptive PLDA algorithm; Once a certain number of voiceprints are collected... p Then, the Adaptive PLDA algorithm is used to update the inter-class and intra-class covariance of PLDA, making the parameters of PLDA more suitable for the use case. The processing steps for the offline branch include: S21. Obtain the voiceprint from the online voiceprint log; S22. Obtain the audio sequence; S23. Extract the Fbank features of the audio sequence; S24. The Fbank features from step S23 are concatenated with the voiceprints of each speaker in the online voiceprint log from step S21 and used as the input to the TS-VAD model. S25: Obtain the results of offline clustering.

2. The flexible offline and online voiceprint logging system according to claim 1, characterized in that, In step S11, the clustering method used is K-means, where each cluster represents a speaker, and the cluster center is assumed to be... voiceprint With cluster center The formula for calculating the log-likelihood ratio is: ; in, , It is the projection matrix of PLDA. The covariance of the PLDA model, The number of voiceprints in the cluster.

3. The flexible offline and online voiceprint logging system according to claim 2, characterized in that, When voiceprint With cluster center When the distance is greater than the threshold, Join the cluster and update the cluster center. Otherwise, the purity of human-generated voiceprints is insufficient, and they will not be added to the cluster to avoid data contamination.

4. The flexible offline and online voiceprint logging system according to claim 1, characterized in that, For overlapping speech, the steps of separating the voiceprints corresponding to the speakers in the overlapping parts, and then clustering the separated voiceprints to obtain the speakers corresponding to the overlapping segments include: S5. Extract speaker representation x for each frame using a voiceprint model. t ; S6. Calculate the voiceprints E of all speakers in overlapping speech. i ; S7, E i ZCA whitening was performed, and then the log-likelihood ratio between the whitened voiceprint and the cluster center was calculated. S8. Clustering: Subsequently, the clustering results of overlapping and non-overlapping speech are integrated to obtain the time annotations of the speakers in the complete audio.

5. A flexible offline / online voiceprint logging system according to claim 4, characterized in that, In step S6, the speaker percentage for each frame is obtained from the EEND model in step S2. Separate the first overlapping speech. Average value per speaker With variance : ； ； Where M is the speaker representation x t The number of frames; Thus, the voiceprint of the i-th speaker in the overlapping speech is obtained. .

Images