Synthesized speech detection method, device, electronic device and storage medium

Abstract

The present invention provides a synthetic speech detection method, device, electronic device and storage medium, wherein the method comprises: determining a speech to be detected; performing speech repair on the speech to be detected to obtain a repaired speech; performing synthetic speech detection based on the repaired speech to obtain a speech detection result, thereby overcoming the defects of the current processed synthetic speech that the detection is difficult and the accuracy is poor, and by restoring the information lost in the processing process, the repaired speech is made as close as possible to the unprocessed pure speech, and detection based on this can more accurately capture the traces of the synthetic speech, thereby greatly improving the accuracy of the synthetic speech detection and the reliability of the detection result, and providing security for the speech-based identity authentication system, as well as providing practical and effective protection for the property and information security of the masses.

Description

Technical Field

[0001] The present invention relates to the field of speech detection technology, and in particular to a synthetic speech detection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of information technology, voice, as an information carrier with unique biological characteristics, plays an important role in the field of identity authentication. However, with the development of deep learning technology, speech synthesis and conversion technology has made great progress, and it can synthesize very realistic speech. These highly realistic false voices will not only deceive the human auditory system, but may also trigger social public opinion and pose a serious threat to the property safety of the masses.

[0003] At present, synthetic speech detection mostly relies on deep learning technology. By applying various deep learning algorithms to synthetic speech detection, the detection efficiency of synthetic speech has been greatly improved. However, the current mainstream synthetic speech detection methods are mostly designed based on clean speech environments, that is, most of them are for detection of pure speech that has not been processed. In actual applications, synthetic speech is often processed in various ways. During the processing, a large amount of speech information is lost, resulting in many traces of synthetic speech being erased, causing a sharp drop in detection performance. Summary of the invention

[0004] The present invention provides a synthetic speech detection method, device, electronic device and storage medium, which are used to solve the defect that synthetic speech detection in the prior art is mostly aimed at pure speech, but performs poorly on processed speech and the detection performance is greatly reduced. By repairing the information lost in the processing process, accurate detection of the processed synthetic speech can be achieved.

[0005] The present invention provides a synthetic speech detection method, comprising: Determine the voice to be detected; Performing voice repair on the voice to be detected to obtain a repaired voice; Perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0006] According to a synthetic speech detection method provided by the present invention, performing speech repair on the speech to be detected to obtain the repaired speech includes: Extracting features of the speech to be detected to obtain speech spectrum features; Inputting the speech spectrum feature into a speech restoration model to obtain a restored speech output by the speech restoration model; The speech restoration model is obtained by conducting adversarial training based on sample compressed speech and sample pure speech corresponding to the sample compressed speech in conjunction with a discriminant model; The discriminant model is used to distinguish the predicted repair speech from the sample clean speech. The predicted repair speech is determined by the speech repair model in the training process based on the sample compressed speech. The sample compressed speech is obtained by lossy compression based on the corresponding sample clean speech.

[0007] According to a synthetic speech detection method provided by the present invention, the speech restoration model is obtained by conducting adversarial training based on the sample compressed speech and its corresponding sample clean speech in conjunction with a discriminant model on the basis of a first predicted classification result of the sample compressed speech and a category label of a sample clean speech corresponding to the sample compressed speech; The category label is used to indicate whether the corresponding sample clean speech is real speech or synthesized speech; The first prediction classification result is used to indicate whether the corresponding sample compressed speech is real speech or synthesized speech; the first prediction classification result is obtained by performing synthesized speech detection based on the predicted repaired speech corresponding to the corresponding sample compressed speech.

[0008] According to a synthetic speech detection method provided by the present invention, the synthetic speech detection is performed based on the repaired speech to obtain a speech detection result, including: Inputting the repaired speech into a classification model to obtain a speech detection speech output by the classification model; The classification model is obtained by training based on the initial classification model, the second predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; the initial classification model is obtained by training based on the sample clean speech and its corresponding category label; The second prediction classification result is used to indicate that the corresponding sample compressed speech is real speech or synthesized speech; the second prediction classification result is obtained by detecting the synthesized speech based on the sample repaired speech corresponding to the corresponding sample compressed speech, and the sample repaired speech is determined by the speech repair model based on the corresponding sample compressed speech.

[0009] According to a synthetic speech detection method provided by the present invention, the speech restoration model and the discrimination model are trained based on the following steps: Constructing an initial speech restoration model and an initial discrimination model; Inputting the sample speech spectrum features of the sample compressed speech into the initial speech restoration model to obtain the predicted restoration speech output by the initial speech restoration model; Input the predicted repaired speech and the sample clean speech corresponding to the sample compressed speech into the initial discriminant model respectively, and obtain the discrimination result of the predicted repaired speech and the sample clean speech output by the initial discriminant model; Based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech, the parameters of the initial speech repair model and the initial discrimination model are updated to obtain the speech repair model and the discrimination model.

[0010] According to a synthetic speech detection method provided by the present invention, the initial speech repair model and the initial discrimination model are updated based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech to obtain the speech repair model and the discrimination model, including: Inputting the predicted repaired speech into an initial classification model to obtain a first predicted classification result of the sample compressed speech output by the initial classification model; Determine a generation loss based on the predicted restoration speech, the sample clean speech, the first predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; Determining the discrimination loss of the initial discrimination model based on the discrimination result of the predicted and repaired speech and the discrimination result of the sample clean speech; Based on the generation loss and the discrimination loss, parameters of the initial speech restoration model and the initial discrimination model are updated to obtain the speech restoration model and the discrimination model.

[0011] According to a synthetic speech detection method provided by the present invention, the classification model is trained based on the following steps: Inputting the sample clean speech into a detection classifier to obtain a predicted classification result of the sample clean speech output by the detection classifier; Based on the predicted classification result of the sample clean speech and the category label of the sample clean speech, updating the parameters of the detection classifier to obtain an initial classification model; Inputting the sample restored speech corresponding to the sample compressed speech into the initial classification model to obtain a second predicted classification result output by the initial classification model; Based on the second predicted classification result and the category label of the sample pure speech corresponding to the sample compressed speech, the parameters of the initial classification model are updated to obtain a classification model.

[0012] According to a synthetic speech detection method provided by the present invention, the sample compressed speech is determined based on the following steps: determining a plurality of compression bit rates; Based on each compression bit rate, lossy compression is performed on the sample clean speech to obtain a sample target speech; Audio format decompression is performed based on the sample target speech to obtain sample compressed speech.

[0013] The present invention also provides a synthetic speech detection device, comprising: A speech determination unit, used for determining the speech to be detected; A speech repair unit, used for performing speech repair on the speech to be detected to obtain a repaired speech; The speech detection unit is used to perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the synthetic speech detection method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the synthetic speech detection method as described above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the synthetic speech detection method as described above is implemented.

[0017] The synthetic speech detection method, device, electronic device and storage medium provided by the present invention perform speech repair on the speech to be detected to obtain the repaired speech, and perform synthetic speech detection based on the repaired speech to obtain the speech detection result, thereby overcoming the defects of the current processed synthetic speech that the detection is difficult and the accuracy is poor, and by restoring the information lost in the processing process, the repaired speech is made as close as possible to the unprocessed pure speech. Based on this, the detection can more accurately capture the traces of the synthetic speech, thereby greatly improving the accuracy of the synthetic speech detection and the reliability of the detection results, and providing security for the speech-based identity authentication system, as well as providing practical and effective protection for the property and information security of the masses. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the synthetic speech detection method provided by the present invention; Figure 2is an example diagram of frequency and amplitude before and after compression provided by the present invention; Figure 3 It is a flow chart of the synthetic speech detection process provided by the present invention; Figure 4 is a schematic diagram of the speech repair model training process provided by the present invention; Figure 5 It is a schematic diagram of the classification model fine-tuning training process provided by the present invention; Figure 6 It is a schematic diagram of the initial classification model training process provided by the present invention; Figure 7 It is a structural schematic diagram of the synthetic speech detection device provided by the present invention; Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Voice, due to its unique biological characteristics, is one of the main ways of identity authentication. For example, the Automatic Speaker Verification (ASV) system uses voice to authenticate the speaker. However, with the development of deep learning technology, text-to-speech (TTS) and voice conversion (VC) technologies are gradually able to generate very realistic voices. These highly realistic false voices can not only easily deceive people's auditory system, but may also trigger social public opinion and even attack various voice-based identity authentication systems, posing a threat to people's property and personal safety.

[0022] To meet this challenge, many research institutions have organized a series of ASV spoof competitions, aiming to pool the wisdom of the public to improve the level of synthetic speech detection. With the widespread application of deep learning algorithms, the effect of synthetic speech detection has been significantly improved, but it still faces many challenges. Specifically, most of the current mainstream detection methods are designed based on clean speech environments, while synthetic speech in actual applications often undergoes various processing, such as lossy compression, channel coding, etc. These processing will cause a large amount of speech information to be lost, and the detection performance will drop sharply.

[0023] The traditional synthetic speech detection algorithm is implemented through two modules: front-end feature extraction and back-end classification. The front-end extracts features, while the back-end uses traditional classifiers such as Gaussian mixture models for classification prediction. With the successful application of neural networks in the field of vision, residual networks, VGG (Visual Geometry Group) networks, and lightweight network structures such as LCNN (LightCNN) have been introduced into back-end classification, effectively improving the detection accuracy. In addition, an end-to-end synthetic speech detection architecture has also been proposed. In this method, when processing discrete speech signals, the first layer of the network is very important, because it not only has to process high-dimensional input signals, but is also most susceptible to the problem of gradient disappearance. At the same time, if the convolutional neural network is used directly for filtering, noise and irregular multi-band information will be learned, which is not only inconsistent with human auditory perception, but also not an effective representation of speech signal features, and it is difficult to guarantee the detection performance of the network.

[0024] It can be seen that although the current synthetic speech detection technology has achieved good results on pure speech, its performance has dropped sharply on data sets processed in various compression formats. In addition, although some results have proposed different features and models to improve the detection results, the effects are not very obvious. The reason is that synthetic speech traces are difficult to detect. After processing, a large amount of speech information will be lost, and synthetic speech traces are difficult to capture, which greatly reduces the detection effect.

[0025] Therefore, how to perform compression-resistant detection on synthetic speech, improve the robustness and generalization of the detection method, and achieve efficient and accurate synthetic speech detection has become a new technical challenge.

[0026] In this regard, the present invention provides a synthetic speech detection method, which aims to solve the problems of high difficulty and poor accuracy in detecting compressed synthetic speech. By restoring the information lost in the compression process, the repaired speech is made as close as possible to the unprocessed pure speech. Detection based on this can more accurately capture traces of synthetic speech, thereby greatly improving the accuracy of synthetic speech detection and the reliability of detection results. Figure 1 is a flow chart of the synthetic speech detection method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, determining the voice to be detected; Step 120, performing voice repair on the detected voice to obtain a repaired voice; Step 130: Perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0027] Considering that the current synthetic speech detection method performs well on pure speech, it still seems to be powerless when dealing with compressed synthetic speech. Figure 2is an example diagram of the frequency and amplitude before and after compression provided by the present invention, such as Figure 2 As shown in the figure, after the speech is compressed in MP3 format, a lot of spectral information is lost, especially the mid-high frequency part, which makes the traces of synthetic speech erased, making it difficult to capture during detection, and the detection effect is greatly reduced.

[0028] Based on this, in the embodiment of the present invention, it is proposed that the compressed speech can be repaired to restore as much original spectrum information as possible, thereby enhancing the detectability of the synthetic speech traces. After that, the synthetic speech detection can be performed based on the repaired speech to achieve accurate recognition of the processed synthetic speech. In the embodiment of the present invention, the synthetic speech detection method based on speech repair breaks the limitations and barriers of traditional detection methods in the detection of processed synthetic speech, bringing new breakthroughs in the field of synthetic speech detection. At the same time, it provides security for the speech-based identity authentication system and provides effective protection for the property and information security of the masses.

[0029] Specifically, before performing synthetic speech detection, it is necessary to first determine the speech to be detected, that is, the speech to be detected. The speech to be detected here can be a synthetic speech generated by speech synthesis, speech conversion and other technologies, or it can be a real person's speech, and the embodiment of the present invention does not make specific limitations. Further, whether it is a synthetic speech or a real speech, it can be an unprocessed pure speech, such as a directly collected speaker's speech, a directly synthesized speech, etc., or it can be a speech that has been processed in various ways, such as a speech that is lossy compressed for a synthetic speech, and a speech that is channel-encoded for a speaker's speech, and the embodiment of the present invention does not make specific limitations.

[0030] However, it should be noted that, since the embodiments of the present invention mainly focus on the problem that there are obvious limitations in the detection of synthesized speech after compression processing, the speech to be detected here is preferably processed speech. Specifically, the speech to be detected can be speech after various processing of the directly collected speaker's speech, or speech after processing of synthesized speech directly synthesized by speech synthesis technology, or speech after processing directly crawled from the Internet, which is not specifically limited in the embodiments of the present invention.

[0031] Furthermore, after determining the speech to be detected, in an embodiment of the present invention, speech repair can be performed on the speech to be detected to obtain repaired speech; here, specifically, effective features of the speech to be detected are first extracted, and then, based on the effective speech features, the speech to be detected is repaired to fill in the missing or damaged information therein, and restore as much spectral information therein as possible, thereby obtaining the repaired speech to be detected, which is referred to as repaired speech. Here, the speech repair process can be implemented by an audio waveform analysis method based on a deep learning algorithm, or by means of currently popular large model technology, or by various speech repair tools, AI (Artificial Intelligence) assistants, or by other methods, which are not specifically limited in the embodiment of the present invention.

[0032] Here, by performing speech repair on the speech to be detected, the missing and damaged information can be filled in, so that it can be restored to its original state as much as possible, and the problem of information loss caused by processing and erasure of synthetic speech traces can be effectively solved, which can greatly enhance the detectability of synthetic speech traces, and then accurately perform synthetic speech detection and optimize the synthetic speech detection effect.

[0033] After that, synthetic speech detection can be performed based on the repaired speech obtained by speech repair to obtain the speech detection result of the speech to be detected. That is, synthetic speech detection is performed on the basis of the repaired speech to be detected, and through the complete frequency, amplitude and other information contained in the repaired speech, it is analyzed to determine whether there are traces of speech synthesis, speech conversion and other technical participation. In other words, it is determined whether there are traces of speech synthesis and whether there are characteristics of synthetic speech, such as too smooth spectrum and unstable fundamental frequency, etc., and the detection result of synthetic speech detection based on the repaired speech is determined according to the judgment result.

[0034] Here, the synthetic speech detection based on the repaired speech can be implemented through an end-to-end classifier, or can be implemented using a deep learning classification model, or can be implemented through a traditional classifier, or can be implemented through other methods, which is not specifically limited in the embodiment of the present invention.

[0035] Figure 3 FIG. 1 is a flow chart of the synthetic speech detection process provided by the present invention, such as Figure 3 As shown, in the embodiment of the present invention, the detection of synthetic speech is not achieved by relying on front-end feature extraction and back-end classification. Instead, speech repair is performed on the speech to be detected to restore the lost information as much as possible, and then efficient classification is performed based on the repaired speech. This avoids the problem that the current detection scheme cannot capture traces of speech synthesis, resulting in extremely difficult detection, low efficiency, and poor results, and achieves efficient and accurate synthetic speech detection.

[0036] The synthetic speech detection method provided by the present invention performs speech repair on the speech to be detected to obtain the repaired speech, and performs synthetic speech detection based on the repaired speech to obtain the speech detection result. The method overcomes the defects of high detection difficulty and poor accuracy of the processed synthetic speech in the traditional scheme, and makes the repaired speech as close as possible to the unprocessed pure speech by restoring the information lost in the processing process. The detection based on this can more accurately capture the traces of the synthetic speech, thereby greatly improving the accuracy of synthetic speech detection, improving the reliability of the detection result, and providing security for the speech-based identity authentication system, as well as providing practical and effective protection for the property and information security of the masses.

[0037] Based on the above embodiment, step 120 includes: Extract features of the speech to be detected to obtain speech spectrum features; Inputting the speech spectrum features into the speech restoration model to obtain the restored speech output by the speech restoration model; The speech restoration model is based on the sample compressed speech and the sample pure speech corresponding to the sample compressed speech, and is obtained by adversarial training with the discriminant model; The discriminant model is used to distinguish between the predicted repair speech and the sample clean speech. The predicted repair speech is determined by the speech repair model in the training process based on the sample compressed speech; the sample compressed speech is obtained by lossy compression based on the corresponding sample clean speech.

[0038] Specifically, in step 120, the process of performing voice repair on the detected voice to obtain the repaired voice may specifically include: First, feature extraction can be performed on the speech to be detected to extract effective spectral features, so as to perform speech repair based on this and restore more spectral information. Here, the speech to be detected can be subjected to Fourier transform first, and the frequency domain signal can be obtained by performing STFT (Short-Time Fourier Transform) on the time domain signal, and then the spectral features can be extracted on this basis, thereby obtaining the speech spectral features of the speech to be detected.

[0039] Here, the speech spectrum feature can be a logarithmic spectrum feature LPS (Log Power Spectrum) after logarithmic operation, or a phase feature based on Fourier transform, such as a modified group delay feature MGDF (Modified Group Delay Function), a cosine phase feature CosPhase (Cosine-phase Function), a relative phase shift feature RPS (Relative Phase Shift), etc. It can also be a cepstral coefficient feature, such as a linear frequency cepstral coefficient feature (Linear Frequency Cepstral Coefficients, LFCC) generated by a linear filter group, a rectangular filter cepstral coefficient feature RFCC (Rectangular Filter Cepstral Coefficients,) obtained by a rectangular filter group, a Mel frequency cepstral coefficient feature MFCC (Mel Frequency Cepstral Coefficients) generated by a Mel filter group and an inverse Mel filter group, and an inverted Mel frequency cepstral coefficient feature IMFCC (Inverted Mel Frequency Cepstral Coefficients), which is not specifically limited in the embodiment of the present invention.

[0040] As a priority, in the embodiment of the present invention, the spectral features are extracted by a Mel filter bank, and the extracted results are logarithmized to obtain the speech spectral features. Here, by extracting spectral features by a Mel filter, which is similar to a triangular window, the extracted features can be made more consistent with the nonlinear response of the human auditory system to sound.

[0041] Subsequently, speech restoration can be performed according to the extracted speech spectrum features to obtain the restored speech, that is, the speech spectrum features are input into the speech restoration model to perform speech restoration on the speech to be detected through the speech restoration model to restore the information lost during the compression process, thereby obtaining the restored speech output by the speech restoration model. Here, specifically, the speech spectrum features can be taken as input and deconvolved in the speech restoration model to realize upsampling operation through the deconvolution stage, so that the lost information in the speech to be detected can be restored, and finally the restored speech after the information is restored is output.

[0042] Before the speech spectrum features of the speech to be detected are input into the speech repair model, in the embodiment of the present invention, the speech repair model can also be pre-trained by using sample compressed speech. Specifically, when training the speech repair model, the sample compressed speech can be used as the training sample, and the sample clean speech corresponding to the sample compressed speech can be used as the training label to perform supervised training, so that the speech repair model can learn the mapping relationship between the sample compressed speech and the sample clean speech during the training process, so that the speech repair model obtained by training can apply the above mapping relationship to perform speech repair on the speech to be detected to generate the corresponding repaired speech, thereby achieving the purpose of speech repair of the compressed speech.

[0043] However, when the speech restoration model obtained by the above-mentioned supervised training method is used for speech restoration, the restored speech may be distorted. Taking this into consideration, the embodiment of the present invention adopts the idea of ​​Generative Adversarial Networks (GAN) to perform adversarial training on the speech restoration model and the discriminant model.

[0044] It should be noted that generative adversarial networks are mostly used in scenarios such as speech synthesis and style transfer. They include a generator network and a discriminator network. The generator is a network that generates speech, and the discriminator is a network that determines the authenticity of speech. The goal of the generator is to generate speech that is as realistic as possible and can confuse the discriminator. The goal of the discriminator is to distinguish the speech generated by the generator from the speech that is not directly input through the generator. The two compete with each other to improve the overall performance of the network and obtain better output results.

[0045] In adversarial training, the speech restoration model can be regarded as a generator, and the discriminant model can be regarded as a discriminator. The speech restoration model as a generator can perform speech restoration on the input sample compressed speech, thereby outputting the restored speech, namely the predicted restored speech; the discriminant model as a discriminator can distinguish the input speech, namely distinguish whether the input speech is the predicted restored speech generated by the generator, or the sample pure speech that is directly input without passing through the generator. In this process, the generator and the discriminator compete with each other. The generator outputs the predicted restored speech as close as possible to the sample pure speech, in order to achieve the purpose of making it difficult for the discriminator to distinguish between the sample pure speech and the predicted restored speech. The discriminator outputs the discrimination result consistent with the actual situation of the input speech, in order to achieve a more accurate and reliable discrimination effect.

[0046] It should be noted that, in the embodiment of the present invention, since the input of the generator may be real speech or synthesized speech, the repaired speech output after the generator repairs it may also be real speech or synthesized speech. When the discriminator makes a judgment, the purpose is not to judge whether the input speech is real speech or synthesized speech, but to judge whether the input speech is the predicted repaired speech generated by the generator or the directly input sample pure speech.

[0047] The sample compressed speech is the speech after lossy compression of the sample clean speech; the sample clean speech is the sample speech that has not been processed, which can be either synthetic speech or real speech. The sample clean speech can be obtained from a public dataset.

[0048] In the embodiment of the present invention, a generation and adversarial mechanism is introduced to train a speech repair model, so that the speech repair ability of the trained speech repair model for the speech to be detected can be guaranteed; moreover, the speech repair model can not only achieve the purpose of speech repair, but also the output repaired speech will be closer to the real uncompressed speech, completely overcoming the defects of high detection difficulty and poor accuracy of processed synthetic speech in traditional solutions, realizing accurate and reliable synthetic speech detection, and optimizing the detection effect.

[0049] Based on the above embodiment, the sample compressed speech is determined based on the following steps: determining a plurality of compression bit rates; Based on each compression bit rate, lossy compression is performed on the sample clean speech to obtain the sample target speech; The audio format is decompressed based on the sample target speech to obtain the sample compressed speech.

[0050] Specifically, the process of determining the sample compressed speech may include: First, a large number of sample clean speech can be obtained from public data sets, such as Chinese synthesized speech, narrowband audio, real speech, etc. from the ASV spoof 2019 data set, and multiple different bit rates, i.e., compression bit rates, can be set to compress the sample clean speech. Here, the compression bit rate can be 32kbps, 64kbps, 128kbps, etc.

[0051] Next, the sample clean speech can be lossily compressed according to the set compression bit rates to obtain the sample target speech; here, specifically, the sample clean speech can be lossily compressed according to the set compression bit rate to obtain compressed speech in a variety of different audio formats (such as mp3, m4a, wma, etc.), which is referred to as the sample target speech; then the sample target speech can be decompressed to decompress it into the set audio format, such as the 16KHz flac audio format, and the speech obtained in this way is the sample compressed speech.

[0052] In the embodiment of the present invention, data enhancement can be achieved through audio compression and decompression processing, which can greatly expand the training samples, so that the model training process has rich training data, which provides support for improving model performance.

[0053] Based on the above embodiment, the speech restoration model is obtained by conducting adversarial training based on the sample compressed speech and its corresponding sample clean speech in conjunction with the discriminant model on the basis of the first predicted classification result of the sample compressed speech and the category label of the sample clean speech corresponding to the sample compressed speech; The category label is used to indicate whether the corresponding sample clean speech is real speech or synthesized speech; The first prediction classification result is used to indicate whether the corresponding sample compressed speech is real speech or synthesized speech; the first prediction classification result is obtained by performing synthesized speech detection based on the predicted repaired speech corresponding to the corresponding sample compressed speech.

[0054] Specifically, in order to enable the speech repair model to perform speech repair better and more accurately, as much information that is beneficial to synthetic speech detection as possible is restored, thereby improving the accuracy of subsequent synthetic speech detection. In an embodiment of the present invention, when training the speech repair model, in addition to the above-mentioned adversarial training loss, classification information can also be introduced to guide the recovery of spectral information in the speech repair process based on the classification result of the predicted repaired speech, so that the speech repair process can develop as much as possible in the direction of restoring spectral information that is beneficial to speech classification / synthetic speech detection, and can tend to restore or even amplify synthetic speech traces that are beneficial to speech classification, thereby improving the detection effect.

[0055] Specifically, in the embodiment of the present invention, based on the loss of adversarial training, combined with the classification loss of speech classification, the generator is guided to update, thereby obtaining a trained generator, that is, a speech repair model. Specifically, when training the speech repair model, after obtaining the predicted repair speech through the generator, the predicted repair speech can be input into the classifier to perform speech classification / synthesized speech detection through the classifier to determine whether it is real speech or synthesized speech, thereby obtaining an output classification result.

[0056] Here, the classifier used for speech classification can be a classification model obtained through supervised training based on the sample pure speech, that is, an initial classification model, or it can be a currently mature classification model that is directly applied, and the embodiment of the present invention does not make specific limitations. At this time, the classification result obtained by speech classification can be called the first predicted classification result, which is actually a category probability distribution, which includes the probability of belonging to synthetic speech (Fake) and the probability of belonging to real speech (Real). Based on the probability, it can be judged whether the sample compressed speech corresponding to the predicted repaired speech is synthetic speech or real speech.

[0057] Furthermore, after obtaining the first predicted classification result, the classification loss can be determined based on the classification result and the category label of the sample clean speech corresponding to the sample compressed speech; here, the category label can indicate whether the sample clean speech is real speech or compressed speech. That is, the loss of the classification process can be determined based on the first predicted classification result of the sample compressed speech and the category label of the sample clean speech corresponding to the sample compressed speech, and on this basis, combined with the loss of adversarial training, the speech restoration model can be trained.

[0058] In an embodiment of the present invention, during the training process of the speech restoration model, classification loss is introduced, and model training is performed in combination with classification loss and adversarial training loss. This can enable the model to develop in the direction of restoring spectral information that is beneficial to speech classification, and tend to restore or even amplify synthetic speech traces that are beneficial to speech classification, thereby improving the detection effect.

[0059] Based on the above embodiment, the speech restoration model and the discrimination model are trained based on the following steps: Constructing an initial speech restoration model and an initial discrimination model; Inputting the sample speech spectrum features of the sample compressed speech into the initial speech restoration model to obtain the predicted restoration speech output by the initial speech restoration model; Input the predicted repaired speech and the sample clean speech corresponding to the sample compressed speech into the initial discriminant model respectively, and obtain the discrimination result of the predicted repaired speech output by the initial discriminant model and the discrimination result of the sample clean speech; Based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech, the parameters of the initial speech repair model and the initial discrimination model are updated to obtain the speech repair model and the discrimination model.

[0060] Specifically, the training process of the speech restoration model and the discrimination model may include the following steps: Figure 4 is a schematic diagram of the speech repair model training process provided by the present invention, such as Figure 4As shown, in the embodiment of the present invention, the initial model of the training process, namely the initial speech repair model and the initial discriminant model, is constructed on the basis of the HIFI-GAN (High Fidelity Generative Adversarial Networks) network. Among them, the initial speech repair model can be regarded as the generator of the generative adversarial network (HIFI-GAN-G), and the initial discriminant model can be regarded as the discriminator (HIFI-GAN-D). It should be noted that the HIFI-GAN architecture includes two discriminators. In the embodiment of the present invention, only one can be selected, or two can be used at the same time to make judgments from different angles.

[0061] Next, the sample speech spectrum features of the sample compressed speech can be input into the initial speech repair model. The initial speech repair model as a generator can perform speech repair according to the input sample speech spectrum features, thereby outputting the predicted repair speech corresponding to the sample compressed speech. Specifically, the generator built based on HIFI-GAN is a full-convolution neural network. After the sample speech spectrum features of the sample compressed speech are input into the initial speech repair model, upsampling can be performed through the deconvolution level until the length of the output sequence matches the time resolution of the waveform of the original sample pure speech, thereby obtaining the predicted repair speech.

[0062] The sample speech spectrum features are obtained by first performing a short-time Fourier transform on the sample compressed speech to obtain a frequency domain signal, and then passing the transformed signal spectrum through a Mel filter bank and taking the logarithm of the result to obtain the sample speech spectrum features. Here, the Mel filter, which is a triangular window-like filter, is more in line with the nonlinear response of the human ear to sound.

[0063] Subsequently, the predicted repaired speech output by the initial speech repair model and the sample clean speech corresponding to the sample compressed speech can be respectively input into the initial discriminant model. The initial discriminant model as a discriminator can discriminate the input predicted repaired speech and the sample clean speech, that is, distinguish whether the input speech is the predicted repaired speech generated by the initial speech repair model (generator) or the sample clean speech directly input without passing through the generator, thereby outputting the discrimination result of the predicted repaired speech and the discrimination result of the sample clean speech.

[0064] Afterwards, the parameters of the initial speech repair model and the initial discrimination model can be adjusted based on the predicted repaired speech and the sample clean speech, as well as the discrimination results of the predicted repaired speech and the sample clean speech, so as to obtain the speech repair model and the discrimination model.

[0065] Here, specifically when adjusting the parameters, for the initial speech repair model, the means can be to output a predicted repaired speech that is as close as possible to the sample clean speech, in order to achieve the purpose that the initial discrimination model has difficulty in distinguishing between the sample clean speech and the predicted repaired speech; for the initial discrimination model, the means can be to output a discrimination result that is consistent with the actual situation of the input speech, in order to achieve a more accurate and reliable discrimination effect.

[0066] Based on the above embodiment, based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech, the parameters of the initial speech repair model and the initial discrimination model are updated to obtain the speech repair model and the discrimination model, including: Inputting the predicted repaired speech into the initial classification model to obtain a first predicted classification result of the sample compressed speech output by the initial classification model; Determine the generation loss based on the first prediction classification results of the predicted repaired speech, the sample clean speech, and the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; Determine the discrimination loss of the initial discrimination model based on the discrimination result of the predicted repaired speech and the discrimination result of the sample pure speech; Based on the generation loss and the discrimination loss, the parameters of the initial speech restoration model and the initial discrimination model are updated to obtain the speech restoration model and the discrimination model.

[0067] Specifically, the process of updating the parameters of the initial speech repair model and the initial discrimination model according to the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech to obtain the speech repair model and the discrimination model may specifically include the following steps: See also Figure 4 It can be seen that in the training process of the speech restoration model, for the input sample compressed speech, after the predicted restoration speech is generated by the generator, in addition to inputting the predicted restoration speech into the discriminator for discrimination, it can also be input into the classifier for speech classification. That is, the predicted restoration speech can be input into the initial classification model to perform classification prediction through the initial classification model to determine its authenticity, whether it is real speech or synthetic speech, and output the predicted classification result, that is, the first predicted classification result of the sample compressed speech.

[0068] After that, the loss of the generator can be determined based on the classification result and the output of the generator, and the loss of the discriminator can be determined based on the discrimination result of the discriminator. Here, specifically, the difference between the predicted repaired speech output by the generator and the sample clean speech corresponding to the sample compressed speech, and the difference between the first predicted classification result output by the classifier and the category label of the sample clean speech corresponding to the sample compressed speech can be determined first; then, based on the difference at these two levels, the loss of the generator in the entire synthetic speech detection task is determined, that is, the generation loss of the initial speech repair model. This loss actually consists of two parts, namely the classification loss and the loss of the generator in adversarial training.

[0069] At the same time, the loss of the discriminator in the speech discrimination task, that is, the discrimination loss of the initial discriminant model, can be determined according to the discrimination result of the predicted repaired speech output by the discriminator and the discrimination result of the sample clean speech corresponding to the sample compressed speech. Here, specifically, the discrimination result of the predicted repaired speech and the discrimination result of the sample clean speech can be judged respectively, and the difference between the discrimination result and the actual situation, that is, whether the discrimination result of the predicted repaired speech and the category label of the sample clean speech are consistent, and whether the discrimination result of the sample clean speech is the same as its category label, and the discrimination loss of the initial discriminant model can be determined according to the judgment result.

[0070] It should be noted that when there are multiple discriminators, the predicted repaired speech and the sample pure speech can be distinguished from different perspectives, and their losses can be weighted to obtain the final discriminant loss.

[0071] Then, the parameters of the initial speech repair model and the initial discriminant model can be adjusted according to the generation loss and the discriminant loss; the specific process is: first, determine the learning rate of the initial speech repair model and the learning rate of the initial discriminant model; then, according to the learning rate of the initial speech repair model, the learning rate of the initial discriminant model, the generation loss of the initial speech repair model, and the discriminant loss of the initial discriminant model, update the parameters of the initial speech repair model and the initial discriminant model, so as to obtain the speech repair model and the discriminant model.

[0072] Based on the above embodiment, the generation loss and the discrimination loss are determined based on the following method: Among them, the generation loss can be calculated by the following formula: The discriminant loss can be calculated by the following formula: In both formulas, To generate the loss, , and They are the initial speech restoration model, the initial discrimination model and the initial classification model. represents sample compressed speech, Express expectations, NLLLoss (NegativeLog-Likelihood Loss). In adversarial training, the discriminator's goal is to judge the speech generated by the generator as 1 and the pure speech of the directly input sample as 0.

[0073] Based on the above embodiment, step 130 includes: Input the repaired speech into the classification model to obtain the speech detection speech output by the classification model; The classification model is trained based on the initial classification model, the second predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; the initial classification model is trained based on the sample clean speech and its corresponding category label; The second prediction classification result is used to indicate that the corresponding sample compressed speech is real speech or synthesized speech; the second prediction classification result is obtained based on synthesized speech detection of the sample repaired speech corresponding to the corresponding sample compressed speech, and the sample repaired speech is determined by the speech repair model based on the corresponding sample compressed speech.

[0074] Specifically, in step 130, the process of performing synthetic speech detection according to the repaired speech to obtain the speech detection result may specifically include: After obtaining the repaired speech corresponding to the speech to be detected, the speech classification / synthesized speech detection can be realized through the classification model based on the repaired speech to obtain the speech detection result of the speech to be detected. That is, the repaired speech can be input into the classification model to perform synthetic speech detection through the classification model. Specifically, the complete frequency, amplitude and other information contained in the repaired speech can be analyzed to determine whether there are traces of speech synthesis and whether there are characteristics of synthetic speech, such as the spectrum is too smooth and the fundamental frequency is not stable enough. Finally, the speech detection result of the speech to be detected output by the classification model can be obtained.

[0075] Before inputting the repaired speech into the classification model for synthetic speech detection, in the embodiment of the present invention, the sample compressed speech and the category label of the sample pure speech corresponding to the sample compressed speech can be used to pre-train the classification model. Figure 5 is a schematic diagram of the classification model fine-tuning training process provided by the present invention, such as Figure 5As shown, after obtaining the trained speech repair model, the input sample compressed speech is subjected to speech repair through the speech repair model to obtain the repaired sample repair speech. It is worth noting that the sample repair speech here is different from the predicted repair speech. It is generated by the trained speech repair model, rather than by the generator in the training process. Subsequently, the sample repair speech output by the speech repair model can be input into the initial classification model to perform speech classification / synthesized speech detection through the initial classification model to determine whether it is real speech or synthesized speech, and output the classification result, that is, the second predicted classification result of the sample compressed speech. Afterwards, the initial classification model can be trained according to the second predicted classification result and the category label of the sample pure speech corresponding to the sample compressed speech to obtain the classification model. Among them, the initial classification model can be obtained by pre-training using the sample pure speech and its corresponding category label.

[0076] Compared with the traditional solution of generalizing detection of synthetic speech in complex environment based on a classifier, in the embodiment of the present invention, the detection process is divided into two steps. First, the speech to be detected is repaired to restore useful information to enhance the synthetic speech traces in the speech and make it easier to capture. Then the generated repaired speech is sent to the synthetic speech detection classifier for detection, thereby reducing the interference of speech information loss on the classifier and improving the detection result. The information loss after compression processing in the traditional solution is avoided, which leads to great detection difficulty, including great difficulty in feature design and classifier design, and poor detection effect, thereby achieving accurate and reliable synthetic speech detection.

[0077] Based on the above embodiment, the classification model is trained based on the following steps: Inputting the sample clean speech into the detection classifier, and obtaining the predicted classification result of the sample clean speech output by the detection classifier; Based on the predicted classification results of the sample clean speech and the category label of the sample clean speech, the parameters of the detection classifier are updated to obtain an initial classification model; Inputting the sample repaired speech corresponding to the sample compressed speech into the initial classification model to obtain a second predicted classification result output by the initial classification model; Based on the second predicted classification result and the category label of the sample pure speech corresponding to the sample compressed speech, the parameters of the initial classification model are updated to obtain a classification model.

[0078] Specifically, the training process of the classification model includes the following steps: At the beginning of classification model training, the initial classification model must be determined first. Figure 6 It is a schematic diagram of the initial classification model training process provided by the present invention, such as Figure 6As shown, in an embodiment of the present invention, an end-to-end SINCNet can be used as a detection classifier, and the classifier to be detected is first trained with a sample clean speech, that is, the sample clean speech is input into the classifier to be detected, so as to perform classification prediction through the classifier to be detected, thereby obtaining the predicted classification result of the sample clean speech output by the classifier to be detected, and the loss of the classifier to be detected when predicting the category of the sample clean speech can be determined based on the predicted classification result of the sample clean speech and the category label of the sample clean speech. The parameters of the detection classifier are updated based on this loss, and the initial classification model can be obtained. Here, the loss function used to measure the loss is NLLLoss, which can measure the difference between the predicted classification result of the model and the actual category label.

[0079] After obtaining the initial classification model, in an embodiment of the present invention, the sample compressed speech generated by the trained speech restoration model can be input into the initial classification model, and speech classification / synthesized speech detection can be performed through the initial classification model to determine whether it is real speech or synthesized speech, and the classification result, i.e., the second predicted classification result of the sample compressed speech, can be output. After that, the initial classification model can be supervised and fine-tuned according to the second predicted classification result to obtain the classification model. That is, the parameters of the initial classification model can be updated according to the second predicted classification result of the sample compressed speech and the category label of the sample pure speech corresponding to the sample compressed speech, and finally a trained classification model can be obtained.

[0080] The synthetic speech detection device provided by the present invention is described below. The synthetic speech detection device described below and the synthetic speech detection method described above can be referred to each other.

[0081] Figure 7 is a schematic diagram of the structure of the synthetic speech detection device provided by the present invention, such as Figure 7 As shown, the device comprises: A speech determination unit 710, configured to determine a speech to be detected; A speech repair unit 720 is used to perform speech repair on the speech to be detected to obtain a repaired speech; The speech detection unit 730 is used to perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0082] The synthetic speech detection device provided by the present invention performs speech repair on the speech to be detected to obtain the repaired speech, and performs synthetic speech detection based on the repaired speech to obtain the speech detection result. It overcomes the defects of high detection difficulty and poor accuracy of the processed synthetic speech in the traditional scheme, and makes the repaired speech as close as possible to the unprocessed pure speech by restoring the information lost in the processing process. The detection based on this can capture the traces of the synthetic speech more accurately, thereby greatly improving the accuracy of synthetic speech detection, improving the reliability of the detection result, and providing security for the speech-based identity authentication system, as well as providing practical and effective protection for the property and information security of the masses.

[0083] Based on the above embodiment, the speech repair unit 720 is used for: Extracting features of the speech to be detected to obtain speech spectrum features; Inputting the speech spectrum feature into a speech restoration model to obtain a restored speech output by the speech restoration model; The speech restoration model is obtained by conducting adversarial training based on sample compressed speech and sample pure speech corresponding to the sample compressed speech in conjunction with a discriminant model; The discriminant model is used to distinguish the predicted repair speech from the sample clean speech. The predicted repair speech is determined by the speech repair model in the training process based on the sample compressed speech. The sample compressed speech is obtained by lossy compression based on the corresponding sample clean speech.

[0084] Based on the above embodiment, the speech restoration model is obtained by conducting adversarial training based on the sample compressed speech and its corresponding sample clean speech in conjunction with a discriminant model on the basis of the first predicted classification result of the sample compressed speech and the category label of the sample clean speech corresponding to the sample compressed speech; The category label is used to indicate whether the corresponding sample clean speech is real speech or synthesized speech; The first prediction classification result is used to indicate whether the corresponding sample compressed speech is real speech or synthesized speech; the first prediction classification result is obtained by performing synthesized speech detection based on the predicted repaired speech corresponding to the corresponding sample compressed speech.

[0085] Based on the above embodiment, the speech detection unit 730 is used for: Inputting the repaired speech into a classification model to obtain a speech detection speech output by the classification model; The classification model is obtained by training based on the initial classification model, the second predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; the initial classification model is obtained by training based on the sample clean speech and its corresponding category label; The second prediction classification result is used to indicate that the corresponding sample compressed speech is real speech or synthesized speech; the second prediction classification result is obtained by detecting the synthesized speech based on the sample repaired speech corresponding to the corresponding sample compressed speech, and the sample repaired speech is determined by the speech repair model based on the corresponding sample compressed speech.

[0086] Based on the above embodiment, the device further includes a model training unit, which is used to: Constructing an initial speech restoration model and an initial discrimination model; Inputting the sample speech spectrum features of the sample compressed speech into the initial speech restoration model to obtain the predicted restoration speech output by the initial speech restoration model; Input the predicted repaired speech and the sample clean speech corresponding to the sample compressed speech into the initial discriminant model respectively, and obtain the discrimination result of the predicted repaired speech and the sample clean speech output by the initial discriminant model; Based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech, the parameters of the initial speech repair model and the initial discrimination model are updated to obtain the speech repair model and the discrimination model.

[0087] Based on the above embodiment, the model training unit is used to: Inputting the predicted repaired speech into an initial classification model to obtain a first predicted classification result of the sample compressed speech output by the initial classification model; Determine a generation loss based on the predicted restoration speech, the sample clean speech, the first predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; Determining the discrimination loss of the initial discrimination model based on the discrimination result of the predicted and repaired speech and the discrimination result of the sample clean speech; Based on the generation loss and the discrimination loss, parameters of the initial speech restoration model and the initial discrimination model are updated to obtain the speech restoration model and the discrimination model.

[0088] Based on the above embodiment, the model training unit is further used for: Inputting the sample clean speech into a detection classifier to obtain a predicted classification result of the sample clean speech output by the detection classifier; Based on the predicted classification result of the sample clean speech and the category label of the sample clean speech, updating the parameters of the detection classifier to obtain an initial classification model; Inputting the sample restored speech corresponding to the sample compressed speech into the initial classification model to obtain a second predicted classification result output by the initial classification model; Based on the second predicted classification result and the category label of the sample pure speech corresponding to the sample compressed speech, the parameters of the initial classification model are updated to obtain a classification model.

[0089] Based on the above embodiment, the model training unit is further used for: determining a plurality of compression bit rates; Based on each compression bit rate, lossy compressing the sample clean speech to obtain a sample target speech; Audio format decompression is performed based on the sample target speech to obtain sample compressed speech.

[0090] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the synthetic speech detection method, which includes: determining a speech to be detected; performing speech repair on the speech to be detected to obtain a repaired speech; performing synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0091] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the synthetic speech detection method provided by the above-mentioned methods, and the method includes: determining a speech to be detected; performing speech repair on the speech to be detected to obtain a repaired speech; and performing synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the synthetic speech detection method provided by the above-mentioned methods. The method includes: determining a speech to be detected; performing speech repair on the speech to be detected to obtain a repaired speech; performing synthetic speech detection based on the repaired speech to obtain a speech detection result.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A synthetic speech detection method, characterized in that: include: Determine the voice to be detected; Performing voice repair on the voice to be detected to obtain a repaired voice; Perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

2. The synthetic speech detection method according to claim 1, characterized in that: The performing voice repair on the to-be-detected voice to obtain the repaired voice includes: Extracting features of the speech to be detected to obtain speech spectrum features; Inputting the speech spectrum feature into a speech restoration model to obtain a restored speech output by the speech restoration model; The speech restoration model is obtained by conducting adversarial training based on sample compressed speech and sample pure speech corresponding to the sample compressed speech in conjunction with a discriminant model; The discriminant model is used to distinguish the predicted repair speech from the sample clean speech. The predicted repair speech is determined by the speech repair model in the training process based on the sample compressed speech. The sample compressed speech is obtained by lossy compression based on the corresponding sample clean speech.

3. The synthetic speech detection method according to claim 2, characterized in that: The speech restoration model is obtained by conducting adversarial training based on the sample compressed speech and its corresponding sample clean speech in conjunction with a discriminant model on the basis of the first predicted classification result of the sample compressed speech and the category label of the sample clean speech corresponding to the sample compressed speech; The category label is used to indicate whether the corresponding sample clean speech is real speech or synthesized speech; The first prediction classification result is used to indicate whether the corresponding sample compressed speech is real speech or synthesized speech; the first prediction classification result is obtained by performing synthesized speech detection based on the predicted repaired speech corresponding to the corresponding sample compressed speech.

4. The synthetic speech detection method according to claim 2, characterized in that: The performing synthetic speech detection based on the repaired speech to obtain a speech detection result includes: Inputting the repaired speech into a classification model to obtain a speech detection speech output by the classification model; The classification model is obtained by training based on the initial classification model, the second predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; the initial classification model is obtained by training based on the sample clean speech and its corresponding category label; The second prediction classification result is used to indicate that the corresponding sample compressed speech is real speech or synthesized speech; the second prediction classification result is obtained by detecting the synthesized speech based on the sample repaired speech corresponding to the corresponding sample compressed speech, and the sample repaired speech is determined by the speech repair model based on the corresponding sample compressed speech.

5. The synthetic speech detection method according to any one of claims 2 to 4, characterized in that: The speech restoration model and the discrimination model are trained based on the following steps: Constructing an initial speech restoration model and an initial discrimination model; Inputting the sample speech spectrum features of the sample compressed speech into the initial speech restoration model to obtain the predicted restoration speech output by the initial speech restoration model; Input the predicted repaired speech and the sample clean speech corresponding to the sample compressed speech into the initial discriminant model respectively, and obtain the discrimination result of the predicted repaired speech and the sample clean speech output by the initial discriminant model; Based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech, the parameters of the initial speech repair model and the initial discrimination model are updated to obtain the speech repair model and the discrimination model.

6. The synthetic speech detection method according to claim 5, characterized in that: The method of updating the parameters of the initial speech repair model and the initial discrimination model based on the predicted repair speech, the sample clean speech, the discrimination result of the predicted repair speech, and the discrimination result of the sample clean speech to obtain the speech repair model and the discrimination model includes: Inputting the predicted repaired speech into an initial classification model to obtain a first predicted classification result of the sample compressed speech output by the initial classification model; Determine a generation loss based on the predicted restoration speech, the sample clean speech, the first predicted classification result of the sample compressed speech, and the category label of the sample clean speech corresponding to the sample compressed speech; Determining the discrimination loss of the initial discrimination model based on the discrimination result of the predicted and restored speech and the discrimination result of the sample clean speech; Based on the generation loss and the discrimination loss, parameters of the initial speech restoration model and the initial discrimination model are updated to obtain the speech restoration model and the discrimination model.

7. The synthetic speech detection method according to claim 4, characterized in that: The classification model is trained based on the following steps: Inputting the sample clean speech into a detection classifier to obtain a predicted classification result of the sample clean speech output by the detection classifier; Based on the predicted classification result of the sample clean speech and the category label of the sample clean speech, updating the parameters of the detection classifier to obtain an initial classification model; Inputting the sample restored speech corresponding to the sample compressed speech into the initial classification model to obtain a second predicted classification result output by the initial classification model; Based on the second predicted classification result and the category label of the sample pure speech corresponding to the sample compressed speech, the parameters of the initial classification model are updated to obtain a classification model.

8. The synthetic speech detection method according to any one of claims 2 to 4, characterized in that: The sample compressed speech is determined based on the following steps: determining a plurality of compression bit rates; Based on each compression bit rate, lossy compression is performed on the sample clean speech to obtain a sample target speech; Audio format decompression is performed based on the sample target speech to obtain sample compressed speech.

9. A synthetic speech detection device, characterized in that: include: A speech determination unit, used to determine the speech to be detected; A speech repair unit, used for performing speech repair on the speech to be detected to obtain a repaired speech; The speech detection unit is used to perform synthetic speech detection based on the repaired speech to obtain a speech detection result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the synthetic speech detection method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the synthetic speech detection method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the synthetic speech detection method according to any one of claims 1 to 8 is implemented.

Images