Synthetic speech identification method, device, system and product

Abstract

The invention relates to the technical field of speech authentication, and discloses a synthetic speech authentication method, device, system and product. The method comprises the following steps: constructing a target data set based on real voice data and synthetic voice data of a plurality of speakers; wherein each piece of voice data has a corresponding speaker tag; constructing an identification model which comprises a feature extraction module, a classification module and a judgment module; training an identification model by using the target data set; after model training is completed, target voice data are processed through the identification model to obtain an identification result; the identification result is used for indicating whether the target voice data is synthetic voice or not. By adopting the method, the accuracy of identifying the synthetic speech can be improved, and the highly simulated synthetic speech of the speaker can be effectively identified.

Description

Technical Field

[0001] The present application relates to the technical field of voice authentication, and in particular to a synthetic voice authentication method, device, system and product. Background Art

[0002] With the rapid development of AI (Artificial Intelligence) technology, speech synthesis technology is becoming increasingly mature. Current speech synthesis technology is mostly used in human-computer interaction, media entertainment, education and training, automobiles and other fields. For example, smart speakers, voice assistants, etc., can realize voice dialogue with users through speech synthesis technology, greatly improving the user experience. In the automotive field, intelligent speech synthesis can realize functions such as in-vehicle navigation and in-vehicle equipment conditions, improving the driver's comfort and driving safety.

[0003] However, with the widespread application of speech synthesis technology, there are also corresponding security issues. For example, in the scenario where personal voiceprint is used as identity authentication, synthesized speech may be forged for reverse attack, impersonating the user's real voice to bypass identity authentication and obtain the user's personal privacy data, device control, etc., causing significant losses to users and related personnel. Therefore, it is very important to effectively identify synthesized speech. Traditional synthetic speech identification technology has a relatively low recognition accuracy when facing highly simulated AI forged synthetic audio, especially for the synthetic speech of speakers with different voice characteristics. It is difficult to effectively identify. Therefore, it is necessary to find a method to accurately identify highly simulated synthetic speech of different speakers. Summary of the invention

[0004] In view of this, the present application aims to propose a synthetic speech identification method, device, system and product to achieve accurate identification of highly simulated synthetic speech.

[0005] To achieve the above purpose, the technical solution of this application is as follows:

[0006] A first aspect of an embodiment of the present application provides a synthetic speech identification method, the method comprising:

[0007] A target data set is constructed based on real speech data and synthesized speech data of multiple speakers, wherein each speech data has a corresponding speaker label;

[0008] Constructing an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of the speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthetic speech based on each speaker category;

[0009] Training a discrimination model using the target dataset;

[0010] After the model training is completed, the target speech data is processed by the identification model to obtain an identification result; the identification result is used to indicate whether the target speech data is synthetic speech.

[0011] Optionally, a target dataset is constructed based on real speech data and synthesized speech data of multiple speakers, including:

[0012] Obtaining real speech data of multiple speakers and corresponding synthesized speech data;

[0013] Adding disturbance noise to the synthesized speech data; the disturbance noise is generated based on the gradient of the target loss function of the discrimination model;

[0014] Add corresponding speaker labels to each real speech data and synthesized speech data;

[0015] The target data set is constructed based on all speech data carrying speaker labels.

[0016] Optionally, using the target data set to train a discrimination model includes:

[0017] Extracting feature vectors of multiple dimensions of each voice data in the target data set through the feature extraction module and generating corresponding fusion vectors;

[0018] By means of the classification module, all fused vectors are clustered to obtain a plurality of speaker categories, and the centroid of each speaker category is determined;

[0019] Constructing a verification set for each speaker; the verification set includes: real speech data and / or synthesized speech data of the speaker;

[0020] Determine the similarity threshold of the corresponding speaker category based on the verification set of each speaker;

[0021] A target loss function is constructed, and the identification model is trained based on the target loss function.

[0022] Optionally, construct an objective loss function, including:

[0023] Constructing a first loss function based on the Euclidean distance and the speaker label; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module;

[0024] Constructing a second loss function based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module;

[0025] The target loss function is constructed based on the first loss function and the second loss function.

[0026] Optionally, the feature vectors of the multiple dimensions include: a spectrum graph, a Mel-frequency cepstral coefficient and a fundamental frequency;

[0027] Extracting feature vectors of multiple dimensions of each speech data in the target data set and generating corresponding fusion vectors specifically includes:

[0028] Performing short-time Fourier transform on the speech data to extract a spectrogram of the speech data;

[0029] Calculating the Mel-frequency energy spectrum of the speech data, and calculating the Mel-frequency cepstrum coefficients based on the Mel-frequency energy spectrum;

[0030] Extract the fundamental frequency of the speech data using the librosa library;

[0031] Convolving the spectrogram, the Mel-frequency cepstral coefficients and the fundamental frequency to generate feature vectors of multiple dimensions;

[0032] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0033] Optionally, clustering all fused vectors to obtain multiple speaker categories and determining the centroid of each speaker category includes:

[0034] Get the preset number of clusters;

[0035] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0036] Using the expectation maximization algorithm, the parameters of the probability density function are determined; the parameters include: weight, mean and variance;

[0037] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding speaker class.

[0038] Optionally, determining a similarity threshold of a corresponding speaker class based on a verification set of each speaker includes:

[0039] For each speaker category, determining a plurality of candidate thresholds;

[0040] Traversing each candidate threshold, and determining an error rate corresponding to each candidate threshold based on a verification set of the speaker;

[0041] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaker class.

[0042] Optionally, determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes:

[0043] Extracting the fusion vector of the speech data in the verification set by the feature extraction module, classifying the fusion vector of the speech data by the classification module, and determining the speaker category closest to the speech data;

[0044] According to the speaker category, obtaining the corresponding centroid and all candidate thresholds;

[0045] By means of the judgment module, the cosine distance between the centroid and the fusion vector is calculated; the cosine distance is compared with each candidate threshold value to obtain a corresponding identification result;

[0046] comparing the identification result with the label of the voice data; if the identification result is inconsistent with the label, determining that the identification result is wrong;

[0047] For each candidate threshold, determining the number of incorrect identification results of all speech data in the verification set;

[0048] An error rate is calculated based on the number of errors in the identification results as the error rate corresponding to the candidate threshold.

[0049] Optionally, the target speech data is processed by the identification model to obtain an identification result, including:

[0050] Extracting feature vectors of multiple dimensions of the target speech data through the feature extraction module and generating a fusion vector;

[0051] Classifying the fusion vector by the classification module to determine the speaker category closest to the target speech data;

[0052] According to the speaker category, obtaining the corresponding centroid and similarity threshold;

[0053] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: when the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the real speech data of the speaker; when the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0054] According to a second aspect of an embodiment of the present application, a synthetic speech identification device is provided, which is used to implement the steps in the method provided in the first aspect of an embodiment of the present application, and the device includes:

[0055] A preprocessing module is configured to construct a target data set based on real speech data and synthesized speech data of multiple speakers; wherein each speech data has a corresponding speaker label; construct an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthesized speech based on each speaker category;

[0056] A training module, configured to train a discrimination model using the target data set;

[0057] The identification module is configured to process the target speech data through the identification model to obtain an identification result after the model training is completed; the identification result is used to indicate whether the target speech data is synthetic speech.

[0058] According to a third aspect of the embodiment of the present application, there is provided a synthetic speech identification system, comprising: the synthetic speech identification device, the data acquisition unit and the speech conversion unit as provided in the second aspect of the embodiment of the present application;

[0059] The data collection unit is used to collect real speech data of multiple speakers;

[0060] The speech conversion unit is configured to generate corresponding transcribed text based on the real speech data; and process the transcribed text using a speech conversion model to generate corresponding synthesized speech data.

[0061] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the first aspect of the embodiment of the present application are implemented.

[0062] According to the fifth aspect of the embodiments of the present application, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the method provided in the first aspect of the embodiments of the present application are implemented.

[0063] Using the synthetic speech identification method provided by the present application, first obtain multiple real speech data and synthetic speech data with speaker labels, and build a target data set based on all speech data. Construct an identification model for identifying whether the speech data is synthetic speech, and the model specifically includes a feature extraction module, a classification module and a judgment module. The feature extraction module is used to extract feature vectors of multiple dimensions from the speech data and process them to generate fusion vectors. The classification module clusters the fusion vectors of all speech data in the target data set to generate multiple speaker categories. In addition, each speaker category has a corresponding centroid and similarity threshold. The judgment module judges any speech data based on the centroid and similarity threshold of each speaker category to identify whether the speech data is the real speech of a certain speaker or synthetic speech.

[0064] The synthetic speech identification method provided in the present application constructs a target data set based on the real speech data and synthetic speech data of different speakers. When extracting the feature vector, the audio features of multiple dimensions are fused to obtain a richer feature representation, and the identification model is trained on this basis so that the model can learn richer speech information, improve the model's recognition ability for complex signals, and thus improve the accuracy of the identification results. Due to the differences in the speech features of different speakers, the traditional detection model using a fixed threshold or a simple classifier is difficult to effectively identify highly simulated AI synthetic speech. The present application forms multiple speaker categories by clustering the fused features of multiple speakers, and then determines a corresponding similarity threshold based on each speaker category. Compared with the traditional method, the identification model of the present application can better adapt to the feature differences of different speakers, improve the accuracy and reliability of the identification model in identifying the authenticity of different speakers' voices, and achieve effective recognition of the speaker's highly simulated AI synthetic speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0066] Figure 1 is a flow chart of a synthetic speech identification method proposed in one embodiment of the present application;

[0067] Figure 2 is a flow chart of generating synthesized speech data in one embodiment of the present application;

[0068] Figure 3 is a flow chart of training a discrimination model in one embodiment of the present application;

[0069] Figure 4 is a schematic diagram of centroid clustering in one embodiment of the present application;

[0070] Figure 5 is a schematic diagram of a synthetic speech identification device proposed in an embodiment of the present application;

[0071] Figure 6 Schematic diagram of a synthetic speech identification system proposed in one embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0074] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0075] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects as detailed in the present application.

[0076] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0077] Traditional speech identification methods mostly use a single method to extract features for analysis and discrimination, and the identification effect on highly simulated AI synthesized speech is poor. For example, speech identification methods based on acoustic feature analysis use traditional acoustic features to distinguish between real speech and synthesized speech based on the differences in the above features. When faced with highly simulated AI synthesized speech, the differences in its acoustic features are very subtle, resulting in low identification accuracy. For example, speech identification methods based on deep learning use deep learning models to automatically learn speech features. The trained model has good recognition capabilities for known synthetic speech types, but cannot effectively deal with synthetic speech generated by new or unknown synthesis methods.

[0078] The present application provides a method that can effectively identify AI synthesized speech, can accurately distinguish the subtle differences between real speech and synthesized speech, can adapt to the differences in voice characteristics of different speakers, and can effectively deal with synthesized speech generated by known or unknown synthesis methods.

[0079] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0080] Figure 1 1 is a flow chart of a synthetic speech identification method proposed in one embodiment of the present application. Figure 1 As shown, the method includes:

[0081] S1: Build a target dataset based on real speech data and synthesized speech data of multiple speakers; each speech data has a corresponding speaker label;

[0082] S2: constructing an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of the speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthetic speech based on each speaker category;

[0083] S3: training a discrimination model using the target dataset;

[0084] S4: After the model training is completed, the target speech data is processed by the identification model to obtain an identification result; the identification result is used to indicate whether the target speech data is synthetic speech.

[0085] In this embodiment, real speech data and corresponding synthesized speech data of multiple different speakers are first obtained, and a target data set is constructed based on all speech data for training the discrimination model. The discrimination model includes a feature extraction module, a classification module, and a judgment module. The feature extraction module is used to extract feature data of multiple dimensions from the speech data, and fuse the feature data of multiple dimensions to generate a fusion vector, so that the model can learn richer speech information and improve the model's recognition ability for complex signals.

[0086] The classification module is used to cluster all fusion vectors in the target data set to obtain multiple speaker categories for subsequent identification of whether the target speech data is synthesized speech. Since the speech features of each speaker are different, clustering and determining the centroid through the classification model can distinguish the feature information of each speaker category. On this basis, determining the corresponding similarity threshold for each speaker category helps the model better adapt to the feature differences of different speakers and improves the accuracy and reliability of the identification model in identifying the authenticity of different speakers' speech.

[0087] The trained identification model is used to process the target speech data to be identified, and the target speech data is classified to estimate the speaker category to which it may belong, and then the target speech data is judged whether it belongs to the speaker category according to the centroid and similarity threshold corresponding to the speaker category, and finally the identification result of the target speech data is obtained. The identification result is that the target speech data is the real speech of the speaker, or the target speech data is synthesized speech.

[0088] This embodiment extracts and fuses audio features of multiple dimensions of speech data to obtain richer feature representations, so that the identification model can learn richer speech information, improve the model's ability to recognize complex signals, and thus improve the accuracy of the identification results. In addition, the fusion features of multiple speakers are clustered to form multiple speaker categories to achieve the purpose of distinguishing the features of different speakers. On this basis, the corresponding similarity threshold is determined based on each speaker category, so that the model can better adapt to the speech features of each speaker, improve the accuracy and reliability of the identification model in identifying the authenticity of different speakers' voices, and achieve effective recognition of the speaker's highly simulated AI synthesized speech.

[0089] As an implementation of the present application, a target data set is constructed based on real speech data and synthesized speech data of multiple speakers, including:

[0090] Obtaining real speech data of multiple speakers and corresponding synthesized speech data;

[0091] Adding disturbance noise to the synthesized speech data; the disturbance noise is generated based on the gradient of the target loss function of the discrimination model;

[0092] Add corresponding speaker labels to each real speech data and synthesized speech data;

[0093] The target data set is constructed based on all speech data carrying speaker labels.

[0094] In one embodiment, fake synthetic speech data is generated using real speech data, and small amplitude perturbations are added to the synthetic speech data to blur the boundary between real and fake audio, thereby enhancing the robustness of the identification model. Figure 2 1 is a flow chart of generating synthesized speech data in one embodiment of the present application. Figure 2 As shown, this embodiment uses two Chinese telephone speech corpora, namely the 873-hour HKUSTMandarin Telephone Speech Corpus and the 15-hour CallHome Mandarin Chinese SpeechCorpus, as real speech data, and inputs the corresponding high-quality transcription text into the speech conversion model to generate forged synthetic speech.

[0095] Optionally, the speech conversion model can select a TTS (Text-to-Speech) model such as PaddleSpeech, Azure Cognitive Services, Alibaba Cloud TTS, etc., which is not limited in this embodiment.

[0096] Traditional methods have the problem of overfitting when training detection models, resulting in insufficient generalization of the model and the inability to accurately identify synthetic speech in different environments. In order to enhance the robustness of the identification model, after generating fake synthetic speech, perturbation noise is added to it to blur the boundary between real audio and fake audio, and the synthetic speech data finally used to train the model is obtained. Specifically, by slightly modifying the input synthetic speech data x to fill the gap between the real speech and the synthetic speech, the final synthetic speech data x is obtained. a for:

[0097] x a =x+n;

[0098]

[0099] Where n is the disturbance noise; α is the degree of control disturbance, which ranges from [-1,1]; y is the target true label of model training, is the predicted label for model training, Represents the gradient of the model's objective loss function with respect to the input data x.

[0100] In this embodiment, according to the speakers corresponding to each real speech data and the synthesized speech data, the corresponding speaker labels are added to each speech data. A target data set is constructed based on the labeled speech data, and the target data set is used to iteratively train the discrimination model. No disturbance noise is added to the synthesized speech data at the beginning of the training (i.e., α=0). As the number of training iterations increases, the degree of disturbance is gradually increased.

[0101] In view of the fact that in the related art, synthetic speech and real speech are directly used for model training without perturbing the synthetic speech, the model is overfitted to a specific speech synthesis mode and lacks generalization ability. In this embodiment, by introducing perturbations based on the gradient of the loss function, the boundary between the real speech and the forged synthetic speech is blurred, thereby enhancing the model's ability to learn subtle differences, enabling the model to learn more robust discriminant features, enabling the identification model to effectively identify the synthetic speech in a complex environment, and the model has stronger generalization ability.

[0102] As an implementation of the present application, using the target data set to train the identification model includes:

[0103] Extracting feature vectors of multiple dimensions of each voice data in the target data set through the feature extraction module and generating corresponding fusion vectors;

[0104] By means of the classification module, all fused vectors are clustered to obtain a plurality of speaker categories, and the centroid of each speaker category is determined;

[0105] Constructing a verification set for each speaker; the verification set includes: real speech data and / or synthesized speech data of the speaker;

[0106] Determine the similarity threshold of the corresponding speaker category based on the verification set of each speaker;

[0107] A target loss function is constructed, and the identification model is trained based on the target loss function.

[0108] Figure 3 : is a flow chart of training a recognition model in one embodiment of the present application. Figure 3As shown, in one embodiment, the training process of the identification model includes: generating disturbed synthetic speech data through high-quality transcription text of real speech data, and constructing a target data set. The target data set is sent to the feature extraction module to extract fusion features, and then clustered by the classification module to obtain different speaker categories, and the centroid of each speaker category is determined. Then, the similarity threshold corresponding to each speaker category is dynamically determined by the judgment module according to the error rate of the training process, and the feature extraction module and the classification module are jointly optimized based on the target loss function to reduce the error rate to a minimum. In this embodiment, based on all speakers corresponding to the speech data in the target data set, the speech data (including real speech and / or synthetic speech) of each speaker is obtained, and the verification sets of each speaker are respectively constructed. The similarity threshold corresponding to the speaker category is determined by the identification result of the verification set.

[0109] After the model training is completed, the target speech data to be identified is input into the identification model, processed by the feature extraction module to obtain the fusion vector, classified by the classification module to obtain the centroid of the closest speaker category, and the judgment module determines whether the target speech data is synthesized speech or the real speech of the speaker based on the similarity threshold between the centroid and the speaker category.

[0110] As an implementation mode of the present application, constructing a target loss function includes:

[0111] Constructing a first loss function based on the Euclidean distance and the speaker label; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module;

[0112] Constructing a second loss function based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module;

[0113] The target loss function is constructed based on the first loss function and the second loss function.

[0114] In one embodiment, a target loss function is constructed based on two different loss functions for training the identification model. Specifically, the target loss function is constructed based on a first loss function and a second loss function.

[0115] In this embodiment, the first loss function L feature The feature extraction loss function is used to optimize the extraction performance of the feature extraction module. By using contrast loss optimization to minimize the distance between similar samples (for example, samples of the same speaker class) and maximize the distance between heterogeneous samples, the specific expression is as follows:

[0116]

[0117] Among them, distance is the Euclidean distance between feature vectors, and y is the output label of the model.

[0118] The second loss function uses the GMM log-likelihood loss function to optimize the clustering performance of the classification module, maximize the log-likelihood of the data, and improve the accuracy of the model clustering. The specific expression is as follows:

[0119]

[0120] Among them, k represents the mixing coefficient of the kth Gaussian component, represents the Gaussian density function.

[0121] The target loss function L is constructed based on the first loss function and the second loss function. The specific expression is as follows:

[0122] L=L feature +λL GMM ; Where λ is the weight of the second loss function.

[0123] In the traditional way, a single loss function is used to train the speech detection model, which can only optimize the performance of one dimension of the model, such as optimizing the classification accuracy of the model. This method has limited performance improvement on the model. In this embodiment, a target loss function is constructed based on two different loss functions, which can take into account both feature extraction performance and classification performance when training the model, thereby greatly improving the overall performance of the model.

[0124] As an implementation manner of the present application, the feature vectors of the multiple dimensions include: a spectrum graph, a Mel-frequency cepstral coefficient, and a fundamental frequency;

[0125] Extracting feature vectors of multiple dimensions of each speech data in the target data set and generating corresponding fusion vectors specifically includes:

[0126] Performing short-time Fourier transform on the speech data to extract a spectrogram of the speech data;

[0127] Calculating the Mel-frequency energy spectrum of the speech data, and calculating the Mel-frequency cepstrum coefficients based on the Mel-frequency energy spectrum;

[0128] Extract the fundamental frequency of the speech data using the librosa library;

[0129] Convolving the spectrogram, the Mel-frequency cepstral coefficients and the fundamental frequency to generate feature vectors of multiple dimensions;

[0130] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0131] In one embodiment, a feature extraction module is used to extract feature vectors of three dimensions in the speech data. By extracting feature vectors of multiple dimensions, the model can learn information of multiple dimensions in the speech data more comprehensively. Specifically, the steps of extracting feature vectors are as follows:

[0132] (1) Extract the spectrogram D. Obtain the speech data and its sampling rate sr, fix the frame length to 40ms, and the frame shift to 20ms. Frame the speech data and record the framed speech data as y(t). Based on the speech data y(t) and the sampling rate sr, use the short-time Fourier transform (STFT) to obtain the spectrogram D, as follows:

[0133]

[0134] Where mT is the frame shift (in seconds); is the time interval between each sample.

[0135] Thus, the calculation of the short-time Fourier transform is directly linked to the sampling rate, ensuring that the time and frequency axes of the spectrogram D are consistent with the sampling rate.

[0136] (2) Extract the Mel frequency cepstral coefficients. Use the Mel filter to calculate the Mel frequency energy spectrum E of the speech data. m , E m After logarithmic transformation, we get L m =log(E m ).

[0137] Then, MFCC (Mel-scale Frequency Cepstral Coefficients) is as follows:

[0138]

[0139] Where M is the number of Mel filters and N is the number of MFCC coefficients required.

[0140] (3) Extract the fundamental frequency. Based on the speech data y(t) and the sampling rate sr, the librosa library in Python is used to extract the fundamental frequency f0 of the audio signal.

[0141] The spectrum graph D, Mel frequency cepstral coefficient MFCC and fundamental frequency f0 are convolved to generate feature vectors of corresponding dimensions. After obtaining feature vectors of three dimensions, the three feature vectors are fused to obtain the corresponding fusion vector, thus completing the vector extraction operation of the speech data.

[0142] Compared with traditional detection schemes, this scheme can fully capture the time domain, frequency domain and acoustic feature information of speech data (such as telephone speech signals) and obtain richer feature representation. By extracting and fusing multiple speech feature vectors with significant identification differences through convolutional neural networks, the model can understand speech signals from multiple perspectives, improve the model's ability to recognize complex signals, and enable the model to effectively recognize synthetic speech from different speakers, different environments and different types.

[0143] In addition, the feature extraction module in this embodiment is extensible, and new feature extraction methods can be easily integrated by increasing the dimension of feature extraction to adapt to new AI synthetic speech technologies that may appear in the future.

[0144] As an implementation of the present application, clustering all fused vectors to obtain multiple speaker categories, and determining the centroid of each speaker category, including:

[0145] Get the preset number of clusters;

[0146] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0147] Using the expectation maximization algorithm, the parameters of the probability density function are determined; the parameters include: weight, mean and variance;

[0148] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding speaker class.

[0149] In one embodiment, the classification module clusters the fusion vectors of the speech data using a Gaussian Mixture Model (GMM) and determines the parameters of the probability density function using an Expectation-Maximization algorithm (EM).

[0150] First, the GMM model is used to cluster the fusion vectors of the real speech data of different speakers. The matrix composed of the real speech feature vectors of different speakers is X = {x1, x2...x N}, where x i Represents the feature vector of the i-th sample. The GMM probability density function is as follows:

[0151]

[0152] Where k is the number of clusters; π i is the weight of the i-th Gaussian component and is the i-th Gaussian distribution; μi is the mean; Σ i is the variance.

[0153] In this embodiment, the speech data of each speaker is clustered to obtain a speaker category, that is, k=N.

[0154] Then, the GMM parameters are estimated using the expectation maximization algorithm, which is divided into two steps: E-step (Expectation Step) and M-step (Maximization Step). The E-step and M-step are iterated repeatedly until the parameters converge or the maximum number of iterations is reached.

[0155] In which, step E calculates each sample x n The probability γ of belonging to each Gaussian component i i (x n ) (i.e. responsibility value):

[0156]

[0157] The M step is used to update the parameters of GMM, including weights, mean and variance:

[0158] (1) Update weight π i :

[0159]

[0160] (2) Update the mean vector μ i :

[0161]

[0162] (3) Update the covariance matrix ∑ i :

[0163]

[0164] Figure 4 Schematic diagram of centroid clustering in one embodiment of the present application. Figure 4 As shown, in this embodiment, a Gaussian mixture model (GMM) is used to cluster different feature clusters (speaker categories), such as Figure 4 There are three speaker categories (feature cluster 1, feature cluster 2, feature cluster 3) shown in the figure. For each speaker category, the mean vector μ of the corresponding Gaussian component is i , as the characteristic centroid of the speaker category. The characteristic centroid of each speaker category is used as the reference centroid in the subsequent training process.

[0165] When the classification module processes the target speech data to be identified, it first merges the vector x with the target speech data point. test Perform classification and calculate the corresponding responsibility value γi (x test ):

[0166]

[0167] Furthermore, by calculating argmax i γ i (x test ) Estimate the speaker category closest to the target speech data. Among them, the argmax() function determines the position corresponding to the element with the maximum value, that is, the speaker category. Based on the closest speaker category, determine the centroid C of the category r (i.e., the feature center of the category). Then, the judgment module calculates the cosine distance d(x) between the centroid and the fusion vector of the target speech data. test , C r ):

[0168]

[0169] The target speech data is determined to be the real speech of the speaker or synthesized speech by comparing the cosine distance with the similarity threshold of the speaker. Specifically, if the cosine distance is less than the similarity threshold, it is determined to be synthesized speech; if the cosine distance is greater than or equal to the similarity threshold, it is determined to be the real speech of the speaker.

[0170] As an implementation of the present application, determining the similarity threshold of the corresponding speaker category based on the verification set of each speaker includes:

[0171] For each speaker category, determining a plurality of candidate thresholds;

[0172] Traversing each candidate threshold, and determining an error rate corresponding to each candidate threshold based on a verification set of the speaker;

[0173] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaker class.

[0174] The validation set of each speaker includes the real audio data and / or synthesized audio data of the speaker. In one embodiment, during the training process, multiple candidate thresholds are obtained by uniformly interval value selection in the value range of [0, 1]. For example, in [0, 1], 100 test thresholds (excluding 0) are obtained by using 0.01 as the interval for value selection.

[0175] In each round of model training, the speaker's validation set is processed using the identification model, and the error rate of the identification result is obtained based on each candidate threshold in a traversal manner. The candidate threshold with the smallest error rate is selected from the candidate thresholds to determine the similarity threshold for the speaker class.

[0176] In this embodiment, by dynamically adjusting the thresholds for different speaker categories during the iterative training process of the model, it is possible to determine the most appropriate judgment thresholds according to the voice characteristics of different speakers on the basis of improving the overall recognition performance of the model, thereby improving the accuracy and reliability of the model in identifying the authenticity of the voices of different speakers.

[0177] As an implementation manner of the present application, determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes:

[0178] Extracting the fusion vector of the speech data in the verification set by the feature extraction module, classifying the fusion vector of the speech data by the classification module, and determining the speaker category closest to the speech data;

[0179] According to the speaker category, obtaining the corresponding centroid and all candidate thresholds;

[0180] By means of the judgment module, the cosine distance between the centroid and the fusion vector is calculated; the cosine distance is compared with each candidate threshold value to obtain a corresponding identification result;

[0181] comparing the identification result with the label of the voice data; if the identification result is inconsistent with the label, determining that the identification result is wrong;

[0182] For each candidate threshold, determining the number of incorrect identification results of all speech data in the verification set;

[0183] An error rate is calculated based on the number of errors in the identification results as the error rate corresponding to the candidate threshold.

[0184] In one embodiment, the error rate corresponding to each candidate threshold is determined in the following manner:

[0185] (1) For each speaker's verification set, extract the fusion vector of the speech data in the verification set, and calculate the responsibility value based on the fusion vector through the classification module to determine the speaker category that the speech data is closest to;

[0186] (2) Obtain the centroid of the speaker class and calculate the cosine distance between the centroid and the fusion vector;

[0187] (3) Traversing all candidate thresholds of the speaker category, comparing the cosine distance with each candidate threshold, and obtaining the corresponding identification result;

[0188] (4) Compare the identification result with the label of the speech data. If the two are inconsistent, the identification result is determined to be wrong. For example, if the identification result is real speech and the label is synthetic speech, the identification result is determined to be wrong;

[0189] (5) For each candidate threshold, the error rate is calculated based on the number of errors in the identification results of all speech data in the validation set:

[0190]

[0191] Among them, FP is the number of synthesized speech that is mistakenly identified as real speech, and FN is the number of real speech that is mistakenly identified as synthesized speech.

[0192] As an implementation mode of the present application, the target speech data is processed by the identification model to obtain an identification result, including:

[0193] Extracting feature vectors of multiple dimensions of the target speech data through the feature extraction module and generating a fusion vector;

[0194] Classifying the fusion vector by the classification module to determine the speaker category closest to the target speech data;

[0195] According to the speaker category, obtaining the corresponding centroid and similarity threshold;

[0196] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: when the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the real speech data of the speaker; when the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0197] In one embodiment, the target speech data to be identified is input into the trained identification model, and the target speech data is processed by the identification model to obtain the identification result, and the process is as follows:

[0198] First, the feature vectors of multiple dimensions of the target speech data are extracted through the feature extraction module, and a fusion vector is generated. Then, the fusion vector is processed by the classification module, the corresponding responsibility value is calculated, and the speaker category closest to the target speech data is further estimated based on the responsibility value. Then, the centroid of the speaker category and the corresponding similarity threshold are obtained. The fusion vector is processed by the judgment module, and the cosine distance between the centroid and the fusion vector is calculated. Finally, the cosine distance is compared with the similarity threshold of the speaker category. When the cosine distance is greater than or equal to the similarity threshold, the target speech data is determined to be the real speech of the speaker; when the cosine distance is less than the similarity threshold, the target speech data is determined to be a forged synthetic speech of the speaker.

[0199] Based on the same inventive concept, an embodiment of the present application provides a synthetic speech identification device. Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a synthetic speech identification device 100 proposed in one embodiment of the present application. Figure 5 As shown, the device comprises:

[0200] The preprocessing module 101 is configured to construct a target data set based on real speech data and synthesized speech data of multiple speakers; wherein each speech data has a corresponding speaker label; construct an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthesized speech based on each speaker category;

[0201] A training module 102, configured to train a discrimination model using the target data set;

[0202] The identification module 103 is configured to process the target speech data through the identification model to obtain an identification result after the model training is completed; the identification result is used to indicate whether the target speech data is synthetic speech.

[0203] As an implementation of the present application, the preprocessing module 101 is configured to construct a target data set based on real speech data and synthesized speech data of multiple speakers, specifically including:

[0204] Obtaining real speech data of multiple speakers and corresponding synthesized speech data;

[0205] Adding disturbance noise to the synthesized speech data; the disturbance noise is generated based on the gradient of the target loss function of the discrimination model;

[0206] Add corresponding speaker labels to each real speech data and synthesized speech data;

[0207] The target data set is constructed based on all speech data carrying speaker labels.

[0208] As an implementation manner of the present application, the training module 102 is configured to train the identification model using the target data set, specifically including:

[0209] Extracting feature vectors of multiple dimensions of each voice data in the target data set through the feature extraction module and generating corresponding fusion vectors;

[0210] By means of the classification module, all fused vectors are clustered to obtain a plurality of speaker categories, and the centroid of each speaker category is determined;

[0211] Constructing a verification set for each speaker; the verification set includes: real speech data and / or synthesized speech data of the speaker;

[0212] Determine the similarity threshold of the corresponding speaker category based on the verification set of each speaker;

[0213] A target loss function is constructed, and the identification model is trained based on the target loss function.

[0214] As an implementation manner of the present application, the training module 102 is configured to construct a target loss function, including:

[0215] Constructing a first loss function based on the Euclidean distance and the speaker label; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module;

[0216] Constructing a second loss function based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module;

[0217] The target loss function is constructed based on the first loss function and the second loss function.

[0218] As an implementation manner of the present application, the feature vectors of the multiple dimensions include: a spectrum graph, a Mel-frequency cepstral coefficient, and a fundamental frequency;

[0219] The training module 102 is configured to extract feature vectors of multiple dimensions of each speech data in the target data set through the feature extraction module and generate corresponding fusion vectors, specifically including:

[0220] Performing short-time Fourier transform on the speech data to extract a spectrogram of the speech data;

[0221] Calculating the Mel-frequency energy spectrum of the speech data, and calculating the Mel-frequency cepstrum coefficients based on the Mel-frequency energy spectrum;

[0222] Extract the fundamental frequency of the speech data using the librosa library;

[0223] Convolving the spectrogram, the Mel-frequency cepstral coefficients and the fundamental frequency to generate feature vectors of multiple dimensions;

[0224] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0225] As an implementation manner of the present application, the training module 102 is configured to cluster all fusion vectors through the classification module to obtain multiple speaker categories and determine the centroid of each speaker category, specifically including:

[0226] Get the preset number of clusters;

[0227] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0228] Using the expectation maximization algorithm, the parameters of the probability density function are determined; the parameters include: weight, mean and variance;

[0229] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding speaker class.

[0230] As an implementation of the present application, the training module 102 is configured to determine the similarity threshold of the corresponding speaker class based on the verification set of each speaker, specifically including:

[0231] For each speaker category, determining a plurality of candidate thresholds;

[0232] Traversing each candidate threshold, and determining an error rate corresponding to each candidate threshold based on a verification set of the speaker;

[0233] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaker class.

[0234] As an implementation manner of the present application, the training module 102 is configured to determine the error rate corresponding to each candidate threshold based on the speaker's verification set, specifically including:

[0235] Extracting the fusion vector of the speech data in the verification set by the feature extraction module, classifying the fusion vector of the speech data by the classification module, and determining the speaker category closest to the speech data;

[0236] According to the speaker category, obtaining the corresponding centroid and all candidate thresholds;

[0237] By means of the judgment module, the cosine distance between the centroid and the fusion vector is calculated; the cosine distance is compared with each candidate threshold value to obtain a corresponding identification result;

[0238] comparing the identification result with the label of the voice data; if the identification result is inconsistent with the label, determining that the identification result is wrong;

[0239] For each candidate threshold, determining the number of incorrect identification results of all speech data in the verification set;

[0240] An error rate is calculated based on the number of errors in the identification results as the error rate corresponding to the candidate threshold.

[0241] As an implementation manner of the present application, the identification module 103 is configured to process the target speech data through the identification model to obtain an identification result, specifically including:

[0242] Extracting feature vectors of multiple dimensions of the target speech data through the feature extraction module and generating a fusion vector;

[0243] Classifying the fusion vector by the classification module to determine the speaker category closest to the target speech data;

[0244] According to the speaker category, obtaining the corresponding centroid and similarity threshold;

[0245] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: when the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the real speech data of the speaker; when the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0246] Based on the same inventive concept, an embodiment of the present application provides a synthetic speech identification system. Figure 6 , Figure 6 Schematic diagram of a synthetic speech identification system proposed in one embodiment of the present application. Figure 6As shown, the system comprises: a synthetic speech identification device, a data acquisition unit and a speech conversion unit as provided in the above embodiments;

[0247] The data collection unit is used to collect real voice data of multiple speakers;

[0248] The speech conversion unit is configured to generate corresponding transcribed text based on the real speech data; and process the transcribed text using a speech conversion model to generate corresponding synthesized speech data.

[0249] In one embodiment, the system includes a data collection unit, a speech conversion unit, and a synthetic speech identification device. The data collection unit collects the original speech of multiple speakers, and the speech conversion unit generates high-quality transcription text based on the original speech, and generates synthetic speech data based on the transcription text. Through the data collection unit, the training samples in the target data set can be continuously updated, such as adding speech data samples of new speakers, or increasing the amount of speech data samples of the original speakers. On this basis, the identification model can be continuously updated, so that the model can cope with the synthetic speech generated by the new speech synthesis method, or the voice characteristics of the new speaker.

[0250] Based on the same inventive concept, an embodiment of the present application provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the synthetic speech identification method described in any of the above embodiments of the present application are implemented.

[0251] Based on the same inventive concept, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the synthetic speech identification method described in any of the above embodiments of the present application are implemented.

[0252] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0253] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0254] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and components involved are not necessarily required by the present application.

[0255] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0256] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0257] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0258] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0259] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application is interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0260] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0261] The above is a detailed introduction to the synthetic speech identification method, device, system and product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A synthetic speech identification method, characterized in that: include: A target data set is constructed based on real speech data and synthesized speech data of multiple speakers, wherein each speech data has a corresponding speaker label; Constructing an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of the speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthetic speech based on each speaker category; Training a discrimination model using the target dataset; After the model training is completed, the target speech data is processed by the identification model to obtain an identification result; the identification result is used to indicate whether the target speech data is synthetic speech.

2. The synthetic speech identification method according to claim 1, characterized in that: The target dataset is constructed based on real speech data and synthetic speech data of multiple speakers, including: Obtaining real speech data of multiple speakers and corresponding synthesized speech data; Adding disturbance noise to the synthesized speech data; the disturbance noise is generated based on the gradient of the target loss function of the discrimination model; Add corresponding speaker labels to each real speech data and synthesized speech data; The target data set is constructed based on all speech data carrying speaker labels.

3. The synthetic speech identification method according to claim 1, characterized in that: The target data set is used to train the identification model, including: Extracting feature vectors of multiple dimensions of each voice data in the target data set through the feature extraction module and generating corresponding fusion vectors; By means of the classification module, all fused vectors are clustered to obtain a plurality of speaker categories, and the centroid of each speaker category is determined; Constructing a verification set for each speaker; the verification set includes: real speech data and / or synthesized speech data of the speaker; Determine the similarity threshold of the corresponding speaker category based on the verification set of each speaker; A target loss function is constructed, and the identification model is trained based on the target loss function.

4. The synthetic speech identification method according to claim 3, characterized in that: Construct the target loss function, including: Constructing a first loss function based on the Euclidean distance and the speaker label; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module; Constructing a second loss function based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module; The target loss function is constructed based on the first loss function and the second loss function.

5. The synthetic speech identification method according to claim 3, characterized in that: The feature vectors of the multiple dimensions include: a spectrum graph, a Mel-frequency cepstral coefficient and a fundamental frequency; Extracting feature vectors of multiple dimensions of each speech data in the target data set and generating corresponding fusion vectors specifically includes: Performing short-time Fourier transform on the speech data to extract a spectrogram of the speech data; Calculating the Mel-frequency energy spectrum of the speech data, and calculating the Mel-frequency cepstrum coefficients based on the Mel-frequency energy spectrum; Extract the fundamental frequency of the speech data using the librosa library; Convolving the spectrogram, the Mel-frequency cepstral coefficients and the fundamental frequency to generate feature vectors of multiple dimensions; The feature vectors of the multiple dimensions are fused to generate a fused vector.

6. The synthetic speech identification method according to claim 3, characterized in that: Cluster all fused vectors to obtain multiple speaker categories and determine the centroid of each speaker category, including: Get the preset number of clusters; Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors; Using the expectation maximization algorithm, the parameters of the probability density function are determined; the parameters include: weight, mean and variance; The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding speaker class.

7. The synthetic speech identification method according to claim 3, characterized in that: Determine the similarity threshold of the corresponding speaker class based on the verification set of each speaker, including: For each speaker category, determining a plurality of candidate thresholds; Traversing each candidate threshold, and determining an error rate corresponding to each candidate threshold based on a verification set of the speaker; In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaker class.

8. The synthetic speech identification method according to claim 7, characterized in that: Determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes: Extracting the fusion vector of the speech data in the verification set by the feature extraction module, classifying the fusion vector of the speech data by the classification module, and determining the speaker category closest to the speech data; According to the speaker category, obtaining the corresponding centroid and all candidate thresholds; By means of the judgment module, the cosine distance between the centroid and the fusion vector is calculated; the cosine distance is compared with each candidate threshold value to obtain a corresponding identification result; comparing the identification result with the label of the voice data; if the identification result is inconsistent with the label, determining that the identification result is wrong; For each candidate threshold, determining the number of incorrect identification results of all speech data in the verification set; An error rate is calculated based on the number of errors in the identification results as the error rate corresponding to the candidate threshold.

9. The synthetic speech identification method according to claim 1, characterized in that: The target speech data is processed by the identification model to obtain an identification result, including: Extracting feature vectors of multiple dimensions of the target speech data through the feature extraction module and generating a fusion vector; Classifying the fusion vector by the classification module to determine the speaker category closest to the target speech data; According to the speaker category, obtaining the corresponding centroid and similarity threshold; The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: when the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the real speech data of the speaker; when the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

10. A synthetic speech identification device, characterized in that: Used to implement the method according to any one of claims 1 to 9, comprising: A preprocessing module is configured to construct a target data set based on real speech data and synthesized speech data of multiple speakers; wherein each speech data has a corresponding speaker label; construct an identification model, including: a feature extraction module, a classification module and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions of speech data in the target data set and generate a fusion vector; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to judge whether any speech data is a synthesized speech based on each speaker category; A training module, configured to train a discrimination model using the target data set; The identification module is configured to process the target speech data through the identification model to obtain an identification result after the model training is completed; the identification result is used to indicate whether the target speech data is synthetic speech.

11. A synthetic speech identification system, characterized in that: include: The synthetic speech identification device, data acquisition unit and speech conversion unit as claimed in claim 9; The data collection unit is used to collect real speech data of multiple speakers; The speech conversion unit is configured to generate corresponding transcription text based on the real speech data; The transcribed text is processed using a speech conversion model to generate corresponding synthesized speech data.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 9 are implemented.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps in the method according to any one of claims 1 to 9 are implemented.

Images