A method, apparatus, system and product for synthesized speech identification

Abstract

This application relates to the field of speech authentication technology, and discloses a method, apparatus, system, and product for identifying synthesized speech. The method includes: constructing a target dataset based on real speech data and synthesized speech data from multiple speakers; wherein each speech data has a corresponding speaker label; constructing an authentication model, including a feature extraction module, a classification module, and a judgment module; training the authentication model using the target dataset; after the model training is completed, processing the target speech data through the authentication model to obtain an authentication result; the authentication result is used to indicate whether the target speech data is synthesized speech. This method can improve the accuracy of synthesized speech authentication and effectively recognize highly realistic synthesized speech from speakers.

Description

Technical Field

[0001] This application relates to the field of speech authentication technology, and in particular to a method, apparatus, system and product for identifying synthesized speech. Background Technology

[0002] With the rapid development of AI (Artificial Intelligence) technology, speech synthesis technology has become increasingly mature. Current speech synthesis technology is widely used in human-computer interaction, media entertainment, education and training, and the automotive industry. For example, smart speakers and voice assistants, through speech synthesis technology, can engage in voice conversations with users, greatly enhancing the user experience. In the automotive field, intelligent speech synthesis can enable functions such as in-vehicle navigation and in-car device status updates, improving driver comfort and driving safety.

[0003] However, the widespread application of speech synthesis technology also presents corresponding security challenges. For example, in scenarios where personal voiceprints are used for identity authentication, synthesized speech can be forged for reverse engineering, impersonating a user's real voice to bypass authentication and obtain the user's personal privacy data, device control, etc., causing significant losses to users and related personnel. Therefore, effective identification of synthesized speech is crucial. Traditional synthesized speech identification technologies have relatively low accuracy when facing highly realistic AI-forged synthesized audio, especially when it is difficult to effectively identify synthesized speech from speakers with different voice characteristics. Therefore, a method is needed to accurately identify highly realistic synthesized speech from different speakers. Summary of the Invention

[0004] In view of this, this application aims to propose a method, device, system and product for synthesized speech identification, so as to achieve accurate identification of highly realistic synthesized speech.

[0005] To achieve the above objectives, the technical solution of this application is as follows:

[0006] A first aspect of this application provides a method for synthesized speech identification, the method comprising:

[0007] The target dataset is constructed based on real speech data and synthesized speech data from multiple speakers; each speech data has a corresponding speaker label.

[0008] A discrimination model is constructed, comprising: a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions from the speech data in the target dataset and generate fusion vectors; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein, each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to determine whether any speech data is synthetic speech based on each speaker category;

[0009] The discrimination model is trained using the target dataset.

[0010] After the model training is completed, the target speech data is processed by the discrimination model to obtain the discrimination result; the discrimination result is used to indicate whether the target speech data is synthetic speech.

[0011] Optionally, a target dataset is constructed based on real speech data and synthesized speech data from multiple speakers, including:

[0012] Acquire real speech data from multiple speakers and their corresponding synthesized speech data;

[0013] Perturbation noise is added to the synthesized speech data; the perturbation noise is generated based on the gradient of the target loss function of the discrimination model;

[0014] Add corresponding speaker tags to each real speech data and synthesized speech data;

[0015] The target dataset is constructed based on all speech data carrying speaker labels.

[0016] Optionally, training the discrimination model using the target dataset includes:

[0017] The feature extraction module extracts feature vectors of multiple dimensions from each speech data in the target dataset and generates corresponding fusion vectors.

[0018] The classification module clusters all fused vectors to obtain multiple speaker categories and determines the centroid of each speaker category.

[0019] Construct a verification set for each speaker; the verification set includes: the speaker's real speech data and / or synthesized speech data;

[0020] Determine the corresponding speaker class similarity threshold based on the validation set of each speaker;

[0021] Construct a target loss function, and train the discrimination model based on the target loss function.

[0022] Optionally, the target loss function is constructed, including:

[0023] A first loss function is constructed based on Euclidean distance and speaker labels; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module.

[0024] A second loss function is constructed based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module.

[0025] The target loss function is constructed based on the first loss function and the second loss function.

[0026] Optionally, the feature vectors of the multiple dimensions include: a spectrogram, Mel frequency cepstral coefficients, and a fundamental frequency;

[0027] Extracting feature vectors from multiple dimensions of each speech data point in the target dataset and generating corresponding fusion vectors, specifically including:

[0028] Perform a short-time Fourier transform on the speech data to extract the spectrogram of the speech data;

[0029] Calculate the Mel frequency energy spectrum of the speech data, and calculate the Mel frequency cepstral coefficients based on the Mel frequency energy spectrum;

[0030] Use the librosa library to extract the fundamental frequency of the speech data;

[0031] Convolve the spectrum, Mel frequency cepstral coefficients, and fundamental frequency to generate a multi-dimensional feature vector;

[0032] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0033] Optionally, clustering is performed on all fused vectors to obtain multiple speaker categories, and the centroid of each speaker category is determined, including:

[0034] Get the pre-set number of clusters;

[0035] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0036] The parameters of the probability density function are determined using the expectation-maximization algorithm; the parameters include: weights, mean, and variance.

[0037] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding human speaker.

[0038] Optionally, a similarity threshold for speaker class is determined based on the validation set of each speaker, including:

[0039] For each speaker category, determine multiple candidate thresholds;

[0040] Iterate through each candidate threshold and determine the error rate corresponding to each candidate threshold based on the speaker's verification set;

[0041] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaking human.

[0042] Optionally, determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes:

[0043] The feature extraction module extracts the fusion vector of the speech data in the validation set, and the classification module classifies the fusion vector of the speech data to determine the speaker category that is closest to the speech data.

[0044] Based on the speaker category, obtain the corresponding centroid and all candidate thresholds;

[0045] The judgment module calculates the cosine distance between the centroid and the fusion vector; the cosine distance is compared with each candidate threshold to obtain the corresponding identification result.

[0046] The identification result is compared with the label of the voice data; if the identification result is inconsistent with the label, the identification result is determined to be incorrect.

[0047] For each candidate threshold, determine the number of instances where the identification result of all speech data in the verification set is incorrect;

[0048] The error rate is calculated based on the number of errors identified, and is used as the error rate corresponding to the candidate threshold.

[0049] Optionally, the discrimination result is obtained by processing the target speech data through the discrimination model, including:

[0050] The feature extraction module extracts feature vectors of multiple dimensions from the target speech data and generates a fusion vector.

[0051] The fusion vector is categorized by the classification module to determine the speaker category that is closest to the target speech data;

[0052] Based on the speaker category, obtain the corresponding centroid and similarity threshold;

[0053] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: if the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the speaker's real speech data; if the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0054] According to a second aspect of the present application, a synthetic speech discrimination apparatus is provided for implementing the steps of the method provided in the first aspect of the present application, the apparatus comprising:

[0055] The preprocessing module is configured to construct a target dataset based on real and synthesized speech data from multiple speakers; each speech data has a corresponding speaker label; and to construct an identification model, including a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract multi-dimensional feature vectors from the speech data in the target dataset and generate fused vectors; the classification module is used to cluster all fused vectors to generate multiple speaker categories; each speaker category has a corresponding centroid and a similarity threshold; and the judgment module is used to determine whether any speech data is synthesized speech based on each speaker category.

[0056] The training module is configured to train the discrimination model using the target dataset;

[0057] The discrimination module is configured to process the target speech data through the discrimination model after the model training is completed to obtain the discrimination result; the discrimination result is used to indicate whether the target speech data is synthetic speech.

[0058] According to a third aspect of the embodiments of this application, a synthetic speech identification system is provided, comprising: a synthetic speech identification device, a data acquisition unit, and a speech conversion unit as provided in the second aspect of the embodiments of this application;

[0059] The data acquisition unit is used to collect real voice data from multiple speakers;

[0060] The speech conversion unit is configured to generate corresponding transcribed text based on the real speech data; and to process the transcribed text using a speech conversion model to generate corresponding synthetic speech data.

[0061] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present application.

[0062] According to a fifth aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps in the method provided in the first aspect of the present application.

[0063] The synthesized speech identification method provided in this application first acquires multiple sets of real and synthesized speech data labeled with speaker tags, and constructs a target dataset based on all the speech data. An identification model is then built to determine whether the speech data is synthesized. This model specifically includes a feature extraction module, a classification module, and a judgment module. The feature extraction module extracts multi-dimensional feature vectors from the speech data and processes them to generate fusion vectors. The classification module clusters the fusion vectors of all speech data in the target dataset to generate multiple speaker categories. Each speaker category has a corresponding centroid and a similarity threshold. The judgment module judges any speech data based on the centroid and similarity threshold of each speaker category to determine whether the speech data is the real speech of a particular speaker or synthesized speech.

[0064] The synthetic speech identification method provided in this application constructs a target dataset based on real and synthetic speech data from different speakers. When extracting feature vectors, it fuses multiple dimensions of audio features to obtain richer feature representations. Based on this, it trains an identification model, enabling the model to learn richer speech information, improving its ability to recognize complex signals, and thus increasing the accuracy of the identification results. Because different speakers have different speech features, traditional detection models using fixed thresholds or simple classifiers are difficult to effectively identify highly realistic AI-synthesized speech. This application, however, clusters the fused features of multiple speakers into multiple speaker categories, and then determines a corresponding similarity threshold for each speaker category. Compared to traditional methods, the identification model in this application can better adapt to the feature differences of different speakers, improving the accuracy and reliability of the identification model in recognizing the authenticity of different speakers' speech, and achieving effective recognition of highly realistic AI-synthesized speech. Attached Figure Description

[0065] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart of a synthetic speech identification method proposed in an embodiment of this application;

[0067] Figure 2 This is a flowchart illustrating the generation of synthesized speech data in one embodiment of this application;

[0068] Figure 3 This is a flowchart of training a discrimination model in one embodiment of this application;

[0069] Figure 4 This is a schematic diagram of centroid clustering in one embodiment of this application;

[0070] Figure 5 This is a schematic diagram of a synthesized speech identification device according to an embodiment of this application;

[0071] Figure 6 This is a schematic diagram of a synthetic speech identification system proposed in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0074] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects as detailed in this application.

[0076] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0077] Traditional speech recognition methods often rely on a single approach to extract features for analysis and discrimination, which proves ineffective against highly realistic AI-synthesized speech. For example, speech recognition methods based on acoustic feature analysis use traditional acoustic features, judging based on the differences between real and synthesized speech. However, these subtle differences in acoustic features mean that highly realistic AI-synthesized speech suffers from low accuracy. Similarly, deep learning-based speech recognition methods utilize deep learning models to automatically learn speech features. While these models demonstrate good recognition capabilities for known types of synthesized speech, they struggle to effectively handle synthesized speech generated by new or unknown methods.

[0078] This application provides a method that can effectively identify AI-synthesized speech, accurately distinguish the subtle differences between real speech and synthesized speech, adapt to the differences in the voice characteristics of different speakers, and effectively deal with synthesized speech generated by known or unknown synthesis methods.

[0079] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0080] Figure 1 This is a flowchart of a synthetic speech identification method proposed in an embodiment of this application. Figure 1 As shown, the method includes:

[0081] S1: Construct the target dataset based on real speech data and synthesized speech data from multiple speakers; where each speech data has a corresponding speaker label;

[0082] S2: Construct an identification model, including: a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions from the speech data in the target dataset and generate fusion vectors; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein, each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to determine whether any speech data is synthetic speech based on each speaker category;

[0083] S3: Train the discrimination model using the target dataset;

[0084] S4: After the model training is completed, the target speech data is processed by the discrimination model to obtain the discrimination result; the discrimination result is used to indicate whether the target speech data is synthetic speech.

[0085] In this embodiment, real speech data and corresponding synthesized speech data from multiple different speakers are first acquired. A target dataset is then constructed based on all the speech data to train the discrimination model. The discrimination model includes a feature extraction module, a classification module, and a judgment module. The feature extraction module extracts multi-dimensional feature data from the speech data and fuses the multi-dimensional feature data to generate a fusion vector. This allows the model to learn richer speech information and improves its ability to recognize complex signals.

[0086] The classification module clusters all fused vectors in the target dataset to obtain multiple speaker categories for subsequent identification of whether the target speech data is synthetic. Since each speaker's speech features are different, clustering and centroid determination through the classification model can distinguish the feature information of each speaker category. Based on this, a corresponding similarity threshold is determined for each speaker category, which helps the model better adapt to the feature differences of different speakers, improving the accuracy and reliability of the identification model in recognizing the authenticity of different speakers' speech.

[0087] The trained discrimination model processes the target speech data to be discriminated against. It categorizes the target speech data to estimate the possible speaker categories it belongs to, and then determines whether the target speech data belongs to that speaker category based on the centroid and similarity threshold corresponding to that speaker category. Finally, it obtains the discrimination result for the target speech data. The discrimination result is either that the target speech data is the speaker's real speech or that the target speech data is synthesized speech.

[0088] This embodiment extracts and fuses audio features from multiple dimensions of speech data to obtain richer feature representations, enabling the discrimination model to learn more speech information, improving its ability to recognize complex signals, and thus increasing the accuracy of the discrimination results. Furthermore, by clustering the fused features of multiple speakers to form multiple speaker categories, it achieves the purpose of distinguishing the features of different speakers. Based on this, a corresponding similarity threshold is determined for each speaker category, allowing the model to better adapt to the speech features of each speaker, improving the accuracy and reliability of the discrimination model in identifying the authenticity of different speakers' speech, and achieving effective recognition of highly realistic AI-synthesized speech.

[0089] As one embodiment of this application, a target dataset is constructed based on real speech data and synthesized speech data from multiple speakers, including:

[0090] Acquire real speech data from multiple speakers and their corresponding synthesized speech data;

[0091] Perturbation noise is added to the synthesized speech data; the perturbation noise is generated based on the gradient of the target loss function of the discrimination model;

[0092] Add corresponding speaker tags to each real speech data and synthesized speech data;

[0093] The target dataset is constructed based on all speech data carrying speaker labels.

[0094] In one embodiment, fake synthetic speech data is generated using real speech data, and small perturbations are added to the synthetic speech data to blur the boundary between real and fake audio, thereby enhancing the robustness of the identification model. Figure 2 This is a flowchart illustrating the generation of synthesized speech data in one embodiment of this application. For example... Figure 2 As shown, this embodiment uses two Chinese telephone speech corpora, HKUSTMandarin Telephone Speech Corpus (873 hours) and CallHomeMandarin Chinese Speech Corpus (15 hours), as real speech data. The corresponding high-quality transcribed text is input into the speech conversion model to generate fake synthetic speech.

[0095] Optionally, the speech conversion model can be any TTS (Text-to-Speech) model such as PaddleSpeech, Azure Cognitive Services, or Alibaba Cloud TTS. This embodiment does not impose any restrictions on this.

[0096] Traditional methods suffer from overfitting during model training, leading to insufficient generalization and an inability to accurately distinguish synthesized speech in different environments. To enhance the robustness of the discrimination model, perturbation noise is added to the generated fake synthesized speech to blur the boundary between real and fake audio, resulting in the final synthesized speech data used for model training. Specifically, the input synthesized speech data x is slightly modified to fill the gap between real and synthesized speech, resulting in the final synthesized speech data x. a for:

[0097] x a =x+n;

[0098]

[0099] Where n represents the perturbation noise; α represents the degree of control perturbation, ranging from [-1, 1]; and y represents the target true label for model training. The predicted labels for model training. This represents the gradient of the model's target loss function with respect to the input data x.

[0100] In this embodiment, speaker labels are added to each piece of real speech data and synthesized speech data according to the corresponding speaker. A target dataset is constructed based on the labeled speech data, and the discrimination model is iteratively trained using the target dataset. At the beginning of training, no perturbation noise is added to the synthesized speech data (i.e., α = 0). As the number of training iterations increases, the perturbation level is gradually increased.

[0101] Given that related technologies directly use synthesized and real speech for model training without perturbing the synthesized speech, the model tends to overfit to specific speech synthesis patterns and lacks generalization ability. In this embodiment, by introducing a perturbation based on the loss function gradient, the boundary between real and forged synthesized speech is blurred, thereby enhancing the model's ability to learn subtle differences. This allows the model to learn more robust discriminative features, enabling the discrimination model to effectively identify synthesized speech in complex environments and giving the model stronger generalization ability.

[0102] As one embodiment of this application, training the discrimination model using the target dataset includes:

[0103] The feature extraction module extracts feature vectors of multiple dimensions from each speech data in the target dataset and generates corresponding fusion vectors.

[0104] The classification module clusters all fused vectors to obtain multiple speaker categories and determines the centroid of each speaker category.

[0105] Construct a verification set for each speaker; the verification set includes: the speaker's real speech data and / or synthesized speech data;

[0106] Determine the corresponding speaker class similarity threshold based on the validation set of each speaker;

[0107] Construct a target loss function, and train the discrimination model based on the target loss function.

[0108] Figure 3 This is a flowchart of training a discrimination model in one embodiment of this application. For example... Figure 3As shown, in one embodiment, the training process of the discrimination model includes: generating perturbed synthetic speech data from high-quality transcribed text of real speech data, and constructing a target dataset. The target dataset is fed into a feature extraction module to extract fused features, and then clustered by a classification module to obtain different speaker categories, determining the centroid of each speaker category. Then, a judgment module dynamically determines the similarity threshold corresponding to each speaker category based on the error rate of the training process, and jointly optimizes the feature extraction module and the classification module based on the target loss function to minimize the error rate. In this embodiment, based on all speakers corresponding to the speech data in the target dataset, speech data (including real speech and / or synthetic speech) of each speaker is obtained, and a validation set for each speaker is constructed. The discrimination results of the validation sets are used to determine the similarity threshold for the corresponding speaker category.

[0109] After the model training is completed, the target speech data to be identified is input into the identification model. The feature extraction module processes the data to obtain a fusion vector, and the classification module classifies the data to obtain the centroid of the closest human type. The judgment module determines whether the target speech data is synthetic speech or the real speech of the speaker based on the similarity threshold between the centroid and the human type.

[0110] As one implementation of this application, constructing a target loss function includes:

[0111] A first loss function is constructed based on Euclidean distance and speaker labels; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module.

[0112] A second loss function is constructed based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module.

[0113] The target loss function is constructed based on the first loss function and the second loss function.

[0114] In one embodiment, a target loss function is constructed based on two different loss functions for discriminating model training. Specifically, the target loss function is constructed based on a first loss function and a second loss function.

[0115] In this embodiment, the first loss function L feature A feature extraction loss function is adopted to optimize the extraction performance of the feature extraction module. By using contrastive loss, the distance between similar samples (e.g., different samples of the same speaker) is minimized, while the distance between dissimilar samples is maximized. The specific expression is as follows:

[0116]

[0117] Where distance is the Euclidean distance between feature vectors, and y is the output label of the model.

[0118] The second loss function uses the GMM log-likelihood loss function to optimize the clustering performance of the classification module, maximize the log-likelihood of the data, and improve the accuracy of model clustering. The specific expression is as follows:

[0119]

[0120] Where k represents the mixing coefficient of the k-th Gaussian component. This represents the Gaussian density function.

[0121] The target loss function L is constructed based on the first and second loss functions, and its specific expression is as follows:

[0122] L = L feature +λL GMM ; where λ is the weight of the second loss function.

[0123] Traditional methods use a single loss function to train speech detection models, which can only optimize performance in one dimension of the model, such as classification accuracy. This approach has limited impact on overall model performance. In this embodiment, however, a target loss function is constructed based on two different loss functions. This allows for simultaneous training of both feature extraction and classification performance, resulting in a significant improvement in the overall model performance.

[0124] As one embodiment of this application, the feature vector of the multiple dimensions includes: a spectrogram, Mel frequency cepstral coefficients, and a fundamental frequency;

[0125] Extracting feature vectors from multiple dimensions of each speech data point in the target dataset and generating corresponding fusion vectors, specifically including:

[0126] Perform a short-time Fourier transform on the speech data to extract the spectrogram of the speech data;

[0127] Calculate the Mel frequency energy spectrum of the speech data, and calculate the Mel frequency cepstral coefficients based on the Mel frequency energy spectrum;

[0128] Use the librosa library to extract the fundamental frequency of the speech data;

[0129] Convolve the spectrum, Mel frequency cepstral coefficients, and fundamental frequency to generate a multi-dimensional feature vector;

[0130] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0131] In one embodiment, a feature extraction module extracts three-dimensional feature vectors from the speech data. By extracting feature vectors of multiple dimensions, the model can learn more comprehensively the information in the speech data across multiple dimensions. Specifically, the steps for extracting feature vectors are as follows:

[0132] (1) Extract the spectrogram D. Acquire the speech data and its sampling rate sr. With a fixed frame length of 40ms and a frame shift of 20ms, divide the speech data into frames and denote the segmented speech data as y(t). Based on the speech data y(t) and the sampling rate sr, use the Short Time Fourier Transform (STFT) to obtain the spectrogram D, as follows:

[0133]

[0134] Where mT is the frame shift (in seconds); It is the time interval for each sample.

[0135] Therefore, the calculation of the short-time Fourier transform is directly related to the sampling rate, ensuring that the time and frequency axes of the spectrum D are consistent with the sampling rate.

[0136] (2) Extract Mel-frequency cepstral coefficients. Calculate the Mel-frequency energy spectrum E of the speech data using a Mel filter. m E m After logarithmic transformation, we get L m =log(E m ).

[0137] Therefore, the MFCC (Mel-scale Frequency Cepstral Coefficients) are as follows:

[0138]

[0139] Where M is the number of Mel filters and N is the number of MFCC coefficients required.

[0140] (3) Extract the fundamental frequency. Based on the speech data y(t) and the sampling rate sr, the fundamental frequency f0 of the audio signal is extracted using the librosa library in Python.

[0141] Convolution operations are performed on the spectrogram D, Mel-frequency cepstral coefficients (MFCC), and fundamental frequency f0 to generate feature vectors of corresponding dimensions. After obtaining the three feature vectors, they are fused to obtain the corresponding fused vector, thus completing the vector extraction operation for the speech data.

[0142] Compared to traditional detection schemes, this scheme can fully capture the time-domain, frequency-domain, and acoustic feature information of speech data (such as telephone voice signals), obtaining richer feature representations. By extracting and fusing multiple speech feature vectors with significant discriminative differences through a convolutional neural network, the model can understand speech signals from multiple perspectives, improving its ability to recognize complex signals and enabling it to effectively recognize synthetic speech from different speakers, in different environments, and of different types.

[0143] Furthermore, the feature extraction module in this embodiment is scalable. By increasing the dimensions of feature extraction, new feature extraction methods can be easily integrated to adapt to new AI-synthesized speech technologies that may emerge in the future.

[0144] As one implementation of this application, clustering is performed on all fused vectors to obtain multiple speaker categories, and the centroid of each speaker category is determined, including:

[0145] Get the pre-set number of clusters;

[0146] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0147] The parameters of the probability density function are determined using the expectation-maximization algorithm; the parameters include: weights, mean, and variance.

[0148] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding human speaker.

[0149] In one embodiment, the classification module uses a Gaussian Mixture Module (GMM) to cluster the fused vectors of the speech data and uses the Expectation-Maximization algorithm (EM) to determine the parameters of the probability density function.

[0150] First, a Generative Model (GMM) is used to cluster the fused vectors of real speech data from different speakers. Let X = {x1, x2, ..., xn} be the matrix composed of the real speech feature vectors of different speakers. N}, where x i Let represent the feature vector of the i-th sample. The GMM probability density function is as follows:

[0151]

[0152] Where k is the number of clusters; π i The weight of the i-th Gaussian component and Let μ be the i-th Gaussian distribution;i Σ is the mean; i Let Variance be the variance.

[0153] In this embodiment, the voice data of each speaker is clustered to obtain a speaker category, that is, let k = N.

[0154] Then, the GMM parameters are estimated using the Expectation-Maximization (EM) algorithm, which consists of two steps: the E-step and the M-step. The E-step and M-step are iteratively performed repeatedly until the parameters converge or the maximum number of iterations is reached.

[0155] In this step, E calculates x for each sample. n The probability γ of belonging to each Gaussian component i i (x n (i.e., responsibility value):

[0156]

[0157] The M-step is used to update the parameters of the GMM, including weights, mean, and variance:

[0158] (1) Update weight π i :

[0159]

[0160] (2) Update the mean vector μ i :

[0161]

[0162] (3) Update the covariance matrix ∑ i :

[0163]

[0164] Figure 4 This is a schematic diagram of centroid clustering in one embodiment of this application. For example... Figure 4 As shown, in this embodiment, a Gaussian Mixture Model (GMM) is used to cluster different feature clusters (speaker categories), such as... Figure 4 The diagram shows three speaker categories (feature cluster 1, feature cluster 2, and feature cluster 3). For each speaker category, the mean vector μ of its corresponding Gaussian components is calculated. i This serves as the feature centroid for that speaker category. The feature centroids of each speaker category are then used as reference centroids in subsequent training processes.

[0165] When the classification module processes the target speech data to be identified, it first fuses the target speech data point vector x. test The data is categorized and the corresponding responsibility value γ is calculated.i (x test ):

[0166]

[0167] Furthermore, by calculating argmax i γ i (x test Estimate the closest speaker category for the target speech data. The `argmax()` function determines the position of the element corresponding to the maximum value, i.e., the speaker category. Based on the closest speaker category, determine the centroid C of that category. r (i.e., the feature center of this category). Then, the cosine distance d(x) between this centroid and the fusion vector of the target speech data is calculated using the judgment module. test C r ):

[0168]

[0169] The cosine distance is compared with a similarity threshold for the speaker's personality to determine whether the target speech data is the speaker's real speech or synthesized speech. Specifically, if the cosine distance is less than the similarity threshold, it is determined to be synthesized speech; if the cosine distance is greater than or equal to the similarity threshold, it is determined to be the speaker's real speech.

[0170] As one implementation of this application, determining the corresponding speaker class similarity threshold based on the verification set of each speaker includes:

[0171] For each speaker category, determine multiple candidate thresholds;

[0172] Iterate through each candidate threshold and determine the error rate corresponding to each candidate threshold based on the speaker's verification set;

[0173] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaking human.

[0174] Each speaker's validation set includes the speaker's real audio data and / or synthetic audio data. In one embodiment, during training, multiple candidate thresholds are obtained by taking values ​​at uniform intervals within the range [0,1]. For example, using 0.01 as the interval within [0,1] yields 100 test thresholds (excluding 0).

[0175] In each round of model training, the discrimination model is used to process the speaker's validation set, and the error rate of the discrimination result is obtained based on each candidate threshold in a traversal manner. The candidate threshold with the smallest error rate is selected from the candidate thresholds and determined as the similarity threshold for that speaker's class.

[0176] In this embodiment, by dynamically adjusting the threshold for different speaker categories during the iterative training of the model, the model can improve its overall recognition performance and determine the most suitable judgment threshold based on the voice characteristics of different speakers, thereby improving the accuracy and reliability of the model in recognizing the authenticity of different speakers' speech.

[0177] As one embodiment of this application, determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes:

[0178] The feature extraction module extracts the fusion vector of the speech data in the validation set, and the classification module classifies the fusion vector of the speech data to determine the speaker category that is closest to the speech data.

[0179] Based on the speaker category, obtain the corresponding centroid and all candidate thresholds;

[0180] The judgment module calculates the cosine distance between the centroid and the fusion vector; the cosine distance is compared with each candidate threshold to obtain the corresponding identification result.

[0181] The identification result is compared with the label of the voice data; if the identification result is inconsistent with the label, the identification result is determined to be incorrect.

[0182] For each candidate threshold, determine the number of instances where the identification result of all speech data in the verification set is incorrect;

[0183] The error rate is calculated based on the number of errors identified, and is used as the error rate corresponding to the candidate threshold.

[0184] In one embodiment, the error rate corresponding to each candidate threshold is determined as follows:

[0185] (1) For each speaker’s validation set, the fusion vector is extracted from the speech data in the validation set, and the responsibility value is calculated based on the fusion vector by the classification module to determine the speaker category that the speech data is closest to.

[0186] (2) Obtain the centroid of the speaker and calculate the cosine distance between the centroid and the fusion vector;

[0187] (3) Traverse all candidate thresholds for the speaker category, compare the cosine distance with each candidate threshold, and obtain the corresponding identification result;

[0188] (4) Compare the identification result with the label of the speech data. If they do not match, the identification result is determined to be incorrect. For example, if the identification result is real speech but the label is synthetic speech, the identification result is determined to be incorrect.

[0189] (5) For each candidate threshold, calculate the error rate based on the number of incorrect identification results for all speech data in the validation set:

[0190]

[0191] Wherein, FP is the number of synthesized speech that was mistakenly identified as real speech, and FN is the number of real speech that was mistakenly identified as synthesized speech.

[0192] As one embodiment of this application, the discrimination result is obtained by processing the target speech data through the discrimination model, including:

[0193] The feature extraction module extracts feature vectors of multiple dimensions from the target speech data and generates a fusion vector.

[0194] The fusion vector is categorized by the classification module to determine the speaker category that is closest to the target speech data;

[0195] Based on the speaker category, obtain the corresponding centroid and similarity threshold;

[0196] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: if the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the speaker's real speech data; if the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0197] In one embodiment, the target speech data to be identified is input into a trained identification model, and the identification model processes the target speech data to obtain the identification result. The process is as follows:

[0198] First, a feature extraction module extracts multi-dimensional feature vectors from the target speech data and generates a fusion vector. Then, a classification module processes this fusion vector, calculates the corresponding responsibility value, and further estimates the closest speaker category based on the responsibility value. Next, the centroid of the speaker's category and the corresponding similarity threshold are obtained. A judgment module processes the fusion vector, calculating the cosine distance between the centroid and the fusion vector. Finally, the cosine distance is compared with the speaker's category similarity threshold. If the cosine distance is greater than or equal to the similarity threshold, the target speech data is determined to be the speaker's genuine speech; if the cosine distance is less than the similarity threshold, the target speech data is determined to be synthesized speech from a fabricated speaker.

[0199] Based on the same inventive concept, one embodiment of this application provides a synthetic speech identification device. (Reference) Figure 5 , Figure 5 This is a schematic diagram of a synthesized speech identification device 100 according to an embodiment of this application. Figure 5 As shown, the device includes:

[0200] The preprocessing module 101 is configured to construct a target dataset based on real speech data and synthesized speech data from multiple speakers; wherein each speech data has a corresponding speaker label; and to construct an identification model, including: a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions from the speech data in the target dataset and generate fusion vectors; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein each speaker category has a corresponding centroid and a similarity threshold; and the judgment module is used to determine whether any speech data is synthesized speech based on each speaker category.

[0201] Training module 102 is configured to train the discrimination model using the target dataset;

[0202] The identification module 103 is configured to process the target speech data through the identification model after the model training is completed to obtain an identification result; the identification result is used to indicate whether the target speech data is synthetic speech.

[0203] As one embodiment of this application, the preprocessing module 101 is configured to construct a target dataset based on real speech data and synthesized speech data from multiple speakers, specifically including:

[0204] Acquire real speech data from multiple speakers and their corresponding synthesized speech data;

[0205] Perturbation noise is added to the synthesized speech data; the perturbation noise is generated based on the gradient of the target loss function of the discrimination model;

[0206] Add corresponding speaker tags to each real speech data and synthesized speech data;

[0207] The target dataset is constructed based on all speech data carrying speaker labels.

[0208] In one embodiment of this application, the training module 102 is configured to train the discrimination model using the target dataset, specifically including:

[0209] The feature extraction module extracts feature vectors of multiple dimensions from each speech data in the target dataset and generates corresponding fusion vectors.

[0210] The classification module clusters all fused vectors to obtain multiple speaker categories and determines the centroid of each speaker category.

[0211] Construct a verification set for each speaker; the verification set includes: the speaker's real speech data and / or synthesized speech data;

[0212] Determine the corresponding speaker class similarity threshold based on the validation set of each speaker;

[0213] Construct a target loss function, and train the discrimination model based on the target loss function.

[0214] In one embodiment of this application, the training module 102 is configured to construct a target loss function, including:

[0215] A first loss function is constructed based on Euclidean distance and speaker labels; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module.

[0216] A second loss function is constructed based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module.

[0217] The target loss function is constructed based on the first loss function and the second loss function.

[0218] As one embodiment of this application, the feature vector of the multiple dimensions includes: a spectrogram, Mel frequency cepstral coefficients, and a fundamental frequency;

[0219] The training module 102 is configured to extract multi-dimensional feature vectors from each speech data in the target dataset and generate corresponding fusion vectors through the feature extraction module, specifically including:

[0220] Perform a short-time Fourier transform on the speech data to extract the spectrogram of the speech data;

[0221] Calculate the Mel frequency energy spectrum of the speech data, and calculate the Mel frequency cepstral coefficients based on the Mel frequency energy spectrum;

[0222] Use the librosa library to extract the fundamental frequency of the speech data;

[0223] Convolve the spectrum, Mel frequency cepstral coefficients, and fundamental frequency to generate a multi-dimensional feature vector;

[0224] The feature vectors of the multiple dimensions are fused to generate a fused vector.

[0225] In one embodiment of this application, the training module 102 is configured to cluster all fusion vectors through the classification module to obtain multiple speaker categories and determine the centroid of each speaker category, specifically including:

[0226] Get the pre-set number of clusters;

[0227] Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors;

[0228] The parameters of the probability density function are determined using the expectation-maximization algorithm; the parameters include: weights, mean, and variance.

[0229] The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding human speaker.

[0230] As one embodiment of this application, the training module 102 is configured to determine the corresponding speaker class similarity threshold based on the validation set of each speaker, specifically including:

[0231] For each speaker category, determine multiple candidate thresholds;

[0232] Iterate through each candidate threshold and determine the error rate corresponding to each candidate threshold based on the speaker's verification set;

[0233] In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaking human.

[0234] In one embodiment of this application, the training module 102 is configured to determine the error rate corresponding to each candidate threshold based on the speaker's verification set, specifically including:

[0235] The feature extraction module extracts the fusion vector of the speech data in the validation set, and the classification module classifies the fusion vector of the speech data to determine the speaker category that is closest to the speech data.

[0236] Based on the speaker category, obtain the corresponding centroid and all candidate thresholds;

[0237] The judgment module calculates the cosine distance between the centroid and the fusion vector; the cosine distance is compared with each candidate threshold to obtain the corresponding identification result.

[0238] The identification result is compared with the label of the voice data; if the identification result is inconsistent with the label, the identification result is determined to be incorrect.

[0239] For each candidate threshold, determine the number of instances where the identification result of all speech data in the verification set is incorrect;

[0240] The error rate is calculated based on the number of errors identified, and is used as the error rate corresponding to the candidate threshold.

[0241] In one embodiment of this application, the identification module 103 is configured to process the target speech data through the identification model to obtain an identification result, specifically including:

[0242] The feature extraction module extracts feature vectors of multiple dimensions from the target speech data and generates a fusion vector.

[0243] The fusion vector is categorized by the classification module to determine the speaker category that is closest to the target speech data;

[0244] Based on the speaker category, obtain the corresponding centroid and similarity threshold;

[0245] The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: if the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the speaker's real speech data; if the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

[0246] Based on the same inventive concept, one embodiment of this application provides a synthetic speech identification system. (Reference) Figure 6 , Figure 6 This is a schematic diagram of a synthetic speech identification system proposed in an embodiment of this application. Figure 6As shown, the system includes: a synthesized speech identification device, a data acquisition unit, and a speech conversion unit as provided in the above embodiments;

[0247] The data acquisition unit is used to collect real voice data from multiple speakers;

[0248] The speech conversion unit is configured to generate corresponding transcribed text based on the real speech data; and to process the transcribed text using a speech conversion model to generate corresponding synthetic speech data.

[0249] In one embodiment, the system includes a data acquisition unit, a speech conversion unit, and a synthesized speech discrimination device. The data acquisition unit acquires raw speech data from multiple speakers, the speech conversion unit generates high-quality transcribed text based on the raw speech, and then generates synthesized speech data based on the transcribed text. The data acquisition unit can continuously update the training samples in the target dataset, for example, by adding speech data samples from new speakers or increasing the number of existing speaker speech data samples. Based on this, the discrimination model can be continuously updated, enabling the model to cope with synthesized speech generated by new speech synthesis methods or the voice characteristics of new speakers.

[0250] Based on the same inventive concept, one embodiment of this application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the synthesized speech identification method as described in any of the above embodiments of this application.

[0251] Based on the same inventive concept, one embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps in the synthesized speech identification method as described in any of the above embodiments of this application.

[0252] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0253] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0254] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and components involved are not necessarily essential to this application.

[0255] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0256] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0257] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0258] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0259] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, this application is to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0260] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0261] The synthesized speech identification method, apparatus, system, and product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying synthesized speech, characterized in that, include: The target dataset is constructed based on real speech data and synthesized speech data from multiple speakers; each speech data has a corresponding speaker label. A discrimination model is constructed, comprising: a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract feature vectors of multiple dimensions from the speech data in the target dataset and generate fusion vectors; the classification module is used to cluster all fusion vectors to generate multiple speaker categories; wherein, each speaker category has a corresponding centroid and similarity threshold; the judgment module is used to determine whether any speech data is synthetic speech based on each speaker category; The discrimination model is trained using the target dataset. After the model training is completed, the target speech data is processed by the discrimination model to obtain the discrimination result; the discrimination result is used to indicate whether the target speech data is synthetic speech.

2. The synthesized speech identification method according to claim 1, characterized in that, The target dataset is constructed based on real speech data from multiple speakers and synthesized speech data, including: Acquire real speech data from multiple speakers and their corresponding synthesized speech data; Perturbation noise is added to the synthesized speech data; the perturbation noise is generated based on the gradient of the target loss function of the discrimination model; Add corresponding speaker tags to each real speech data and synthesized speech data; The target dataset is constructed based on all speech data carrying speaker labels.

3. The synthesized speech identification method according to claim 1, characterized in that, Training the discrimination model using the target dataset includes: The feature extraction module extracts feature vectors of multiple dimensions from each speech data in the target dataset and generates corresponding fusion vectors. The classification module clusters all fused vectors to obtain multiple speaker categories and determines the centroid of each speaker category. Construct a verification set for each speaker; the verification set includes: the speaker's real speech data and / or synthesized speech data; Determine the corresponding speaker class similarity threshold based on the validation set of each speaker; Construct a target loss function, and train the discrimination model based on the target loss function.

4. The synthesized speech identification method according to claim 3, characterized in that, Construct the target loss function, including: A first loss function is constructed based on Euclidean distance and speaker labels; the first loss function is used to optimize the feature extraction accuracy of the feature extraction module. A second loss function is constructed based on Gaussian density; the second loss function is used to optimize the clustering accuracy of the classification module. The target loss function is constructed based on the first loss function and the second loss function.

5. The synthesized speech identification method according to claim 3, characterized in that, The multi-dimensional feature vectors include: spectrogram, Mel frequency cepstral coefficients, and fundamental frequency; Extracting feature vectors from multiple dimensions of each speech data point in the target dataset and generating corresponding fusion vectors, specifically including: Perform a short-time Fourier transform on the speech data to extract the spectrogram of the speech data; Calculate the Mel frequency energy spectrum of the speech data, and calculate the Mel frequency cepstral coefficients based on the Mel frequency energy spectrum; Use the librosa library to extract the fundamental frequency of the speech data; Convolve the spectrum, Mel frequency cepstral coefficients, and fundamental frequency to generate a multi-dimensional feature vector; The feature vectors of the multiple dimensions are fused to generate a fused vector.

6. The synthesized speech identification method according to claim 3, characterized in that, Clustering of all fused vectors yields multiple speaker categories, and the centroids of each speaker category are determined, including: Get the pre-set number of clusters; Based on the number of clusters, a probability density function is constructed using a Gaussian mixture model to cluster all fused vectors; The parameters of the probability density function are determined using the expectation-maximization algorithm; the parameters include: weights, mean, and variance. The mean of each Gaussian component in the probability density function is determined as the centroid of the corresponding human speaker.

7. The synthesized speech identification method according to claim 3, characterized in that, Based on the validation sets of each speaker, the corresponding speaker class similarity threshold is determined, including: For each speaker category, determine multiple candidate thresholds; Iterate through each candidate threshold and determine the error rate corresponding to each candidate threshold based on the speaker's verification set; In each round of training, the candidate threshold corresponding to the lowest error rate is determined as the similarity threshold of the speaking human.

8. The synthesized speech identification method according to claim 7, characterized in that, Determining the error rate corresponding to each candidate threshold based on the speaker's verification set includes: The feature extraction module extracts the fusion vector of the speech data in the validation set, and the classification module classifies the fusion vector of the speech data to determine the speaker category that is closest to the speech data. Based on the speaker category, obtain the corresponding centroid and all candidate thresholds; The judgment module calculates the cosine distance between the centroid and the fusion vector; the cosine distance is compared with each candidate threshold to obtain the corresponding identification result. The identification result is compared with the label of the voice data; if the identification result is inconsistent with the label, the identification result is determined to be incorrect. For each candidate threshold, determine the number of instances where the identification result of all speech data in the verification set is incorrect; The error rate is calculated based on the number of errors identified, and is used as the error rate corresponding to the candidate threshold.

9. The synthesized speech identification method according to claim 1, characterized in that, The discrimination result is obtained by processing the target speech data through the discrimination model, including: The feature extraction module extracts feature vectors of multiple dimensions from the target speech data and generates a fusion vector. The fusion vector is categorized by the classification module to determine the speaker category that is closest to the target speech data; Based on the speaker category, obtain the corresponding centroid and similarity threshold; The judgment module calculates the cosine distance between the centroid and the fusion vector; compares the cosine distance with the similarity threshold to obtain an identification result, including: if the cosine distance is greater than or equal to the similarity threshold, determining that the target speech data is the speaker's real speech data; if the cosine distance is less than the similarity threshold, determining that the target speech data is synthesized speech data.

10. A synthesized speech identification device, characterized in that, For implementing the method as described in any one of claims 1-9, comprising: The preprocessing module is configured to construct a target dataset based on real and synthesized speech data from multiple speakers; each speech data has a corresponding speaker label; and to construct an identification model, including a feature extraction module, a classification module, and a judgment module; the feature extraction module is used to extract multi-dimensional feature vectors from the speech data in the target dataset and generate fused vectors; the classification module is used to cluster all fused vectors to generate multiple speaker categories; each speaker category has a corresponding centroid and a similarity threshold; and the judgment module is used to determine whether any speech data is synthesized speech based on each speaker category. The training module is configured to train the discrimination model using the target dataset; The discrimination module is configured to process the target speech data through the discrimination model after the model training is completed to obtain the discrimination result; the discrimination result is used to indicate whether the target speech data is synthetic speech.

11. A synthetic speech recognition system, characterized in that, include: The synthesized speech identification device, data acquisition unit, and speech conversion unit as described in claim 9; The data acquisition unit is used to collect real voice data from multiple speakers; The speech conversion unit is configured to generate corresponding transcribed text based on the real speech data; The transcribed text is processed using a speech conversion model to generate corresponding synthesized speech data.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-9.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps in the method as described in any one of claims 1-9.

Images