A synthetic speech detection method and system based on generative adversarial network

Abstract

The present invention belongs to the field of network security-synthesized speech detection, and specifically relates to a synthetic speech detection method and system based on a generative adversarial network. The method comprises obtaining speech data to be detected and preprocessing the speech data; detecting the preprocessed speech data using a trained speech detection model to obtain detection results, and classifying and labeling the detection results. The speech detection model is a generative adversarial network. The present invention utilizes the game learning concept of the generative adversarial network model to ultimately achieve good synthetic speech detection results. It combines natural language processing technology to perform word segmentation and sentiment analysis on the desired text. An encoder is then used to extract features from phonetic characters to obtain hidden features, which are then fed into a multi-head attention mechanism. Finally, the hidden features are decoded to reconstruct a synthetic speech vector that is closer to the target object, thereby improving detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of network security-synthetic speech detection, and in particular relates to a synthetic speech detection method and system based on a generative adversarial network. Background Art

[0002] With the rapid development of network technology and the increasingly close connection between the Internet economy and people's lives, people are enjoying the great convenience and speed brought by the Internet on the one hand, but are also suffering from the threat of telecommunications fraud on the other.

[0003] Speech synthesis uses deep learning networks to learn speech materials and output speech clips that simulate the target's specific content. Synthetic speech applies new technologies such as artificial intelligence, machine learning, and big data mining to fraud, making fraud more intelligent, cost-effective, and covert, and the dissemination of fraudulent information more precise and effective. Compared to traditional telecommunications fraud, synthetic speech fraud has the following new characteristics:

[0004] 1. More deceptive and confusing.

[0005] 2. The fraudulent methods are highly technological and difficult for the human ear to distinguish. Synthesized speech is created through high-tech means such as deep learning and AI. Human senses alone cannot distinguish true from false, requiring technical means to verify authenticity. Summary of the Invention

[0006] To solve the problems existing in the above-mentioned prior art, the present invention proposes a synthetic speech detection method based on a generative adversarial network, which comprises: obtaining speech data to be detected and preprocessing the speech data; using a trained speech detection model to detect the preprocessed speech data, obtaining detection results, and classifying and labeling the detection results; the speech detection model is a generative adversarial network;

[0007] The process of training a speech detection model includes:

[0008] S1: Obtain a training set, where the data includes expected text and voice materials;

[0009] S2: Perform natural language sentiment analysis on the expected text in the training set to obtain phonetic characters; natural language processing on the expected text includes lexical analysis, syntactic analysis, semantic analysis, and context analysis to obtain sentiment words, and convert the sentiment words into phonetic characters;

[0010] S3: The first convolutional neural network is used to extract features from Zhuyin characters, and the extracted features are converted into hidden vectors through the encoder; the hidden vectors are processed using a multi-head attention mechanism;

[0011] S4: Decode the hidden vector after the multi-head attention mechanism and input the decoded vector into the second convolutional neural network to obtain the speech acoustic features;

[0012] S5: Convert speech acoustic features into time domain waveforms and synthesize them into speech information;

[0013] S6: Mixing the speech information with the corresponding speech material, and performing preliminary feature extraction on the mixed speech information using symmetric Mel-frequency cepstral coefficients;

[0014] S7: Use a multi-scale deep convolutional neural network to perform deep feature extraction on the preliminary features to obtain multi-scale feature information;

[0015] S8: Combine multi-scale features to obtain detection results;

[0016] S9: Calculate the loss function of the model based on the detection results, continuously adjust the model parameters, and complete the model training when the loss function converges.

[0017] Preferably, the process of preprocessing the speech data includes: performing natural language sentiment analysis, pre-emphasis, framing and windowing processing on the speech data; pre-emphasis is multiplying the speech data by a coefficient in the frequency domain, and the coefficient is positively correlated with the frequency of the speech data; framing processing includes segmenting the pre-emphasized speech data in combination with the natural language processing sentiment analysis results to obtain short-time speech data; windowing processing includes multiplying the short-time speech data by a window function to obtain windowed short-time speech data.

[0018] A synthetic speech detection system based on a generative adversarial network, the system comprising: a speech data acquisition module, a speech preprocessor, a generative adversarial network model trainer, and a detection classifier;

[0019] The voice data acquisition module is used to acquire the voice data to be detected;

[0020] The speech preprocessor is used to preprocess the input speech data; wherein the speech preprocessor includes a natural language processing emotion analysis module, a pre-emphasis module, a framing module, and a windowing module;

[0021] The natural language processing sentiment analysis module performs lexical analysis, syntactic analysis, semantic analysis and context analysis on the voice data, completes word segmentation and sentiment analysis, and weights the voice data containing emotional tone to facilitate subsequent module processing;

[0022] The pre-emphasis module is used to multiply the voice data by the coefficient in the frequency domain to compensate for the high-frequency part of the voice signal suppressed by the pronunciation system;

[0023] The framing module is used to segment the pre-emphasized voice data to obtain short-term voice data;

[0024] The windowing module is used to multiply the short-term speech data by the window function to obtain the windowed short-term speech data;

[0025] The generative adversarial network model trainer is used to train the speech detection model to obtain a trained speech detection model;

[0026] The detection classifier includes a speech detection module and a result classification module;

[0027] The speech detection module is a trained speech detection model, which processes the windowed short-term speech data to obtain the detection result;

[0028] The result classification module classifies and marks the detection results.

[0029] Preferably, the generative adversarial network model trainer includes a speech generation module, a speech discrimination module and a back propagation module;

[0030] The speech generation module includes a text conversion unit, an encoding unit, a decoding unit and a sound synthesis unit;

[0031] The text conversion unit converts long text paragraphs or sentences into more detailed word and character levels through a matching algorithm; and then converts a long string of text characters into phonetic characters based on specific semantic information;

[0032] The encoding unit encodes the converted phonetic characters into latent vectors after passing them through several layers of convolutional networks;

[0033] The decoding unit recalculates the above code to obtain a new code through several layers of LSTM network, then obtains the output spectrum frame through the linear mapping layer, and finally obtains the speech acoustic characteristics through several layers of convolutional network;

[0034] The sound synthesis unit converts the above-mentioned speech acoustic features into a time domain waveform to obtain speech information;

[0035] The speech recognition module includes a feature extraction unit, a depth convolution unit and a detection output unit;

[0036] The feature extraction unit uses symmetric Mel-frequency cepstral coefficients to complete preliminary feature extraction, then performs further feature extraction through several layers of convolutional networks, and finally outputs the results to the deep convolution unit as the input of the unit;

[0037] The deep convolution unit uses several layers of deep convolutional networks with different array convolution kernel sizes to further extract features, and then sends the features extracted by these groups of networks to the pooling layer to complete the pooling operation, and then splices these groups of pooling results to obtain higher-dimensional information;

[0038] The detection and output unit uses the feature information obtained by the splicing to complete the discrimination and classification of the speech using a fully connected layer, classifies the input speech into one of the two categories of synthesized speech and non-synthesized speech, and outputs the result;

[0039] The back propagation module compares the results output by the speech discrimination module with the correct results, constructs a loss function, and backpropagates to the speech generation module and the speech discrimination module to optimize the network structure parameters within the above modules, thereby improving the synthesis quality of the generation module and the detection effect of the discrimination module.

[0040] Beneficial effects of the present invention:

[0041] This invention can effectively address the increasingly serious problem of synthetic voice telecommunications fraud. Technically, it utilizes the game learning concept of the generative adversarial network model to ultimately achieve good synthetic voice detection results. It combines natural language processing technology to perform word segmentation and sentiment analysis on the expected text. Secondly, it uses an encoder to extract features from phonetic characters to obtain hidden features, which are then fed into a multi-head attention mechanism. Finally, the hidden features are decoded to reconstruct a synthetic voice vector that is closer to the target object. In addition, two general generative adversarial networks are used to form a new ring generation network, and based on this, a new loss function is constructed, which ultimately improves the detection effect of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of the system structure of an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the structure of a cyclic generative adversarial network of the present invention;

[0044] Figure 3 is a schematic diagram of a speech processor of the present invention;

[0045] Figure 4 Schematic diagram of a generative adversarial network model trainer of the present invention;

[0046] Figure 5 Schematic diagram of the speech generation module of the generative adversarial network model trainer of the present invention;

[0047] Figure 6 Schematic diagram of the speech discrimination module of the adversarial network model trainer of the present invention;

[0048] Figure 7It is a detection classification schematic diagram of the present invention;

[0049] Figure 8 is a schematic diagram of the system structure of another embodiment of the present invention;

[0050] Figure 9 is a schematic diagram of a generative adversarial network model trainer according to another embodiment of the present invention;

[0051] Figure 10 It is a detection classification schematic diagram of another embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] A synthetic speech detection method based on a generative adversarial network (GAN) is disclosed. The method comprises: obtaining speech data to be detected and preprocessing the speech data; detecting the preprocessed speech data using a trained speech detection model to obtain detection results, and classifying and labeling the detection results. The speech detection model is a GAN.

[0054] The process of training a speech detection model includes:

[0055] S1: Obtain a training set, where the data includes expected text and voice materials;

[0056] S2: Perform natural language processing on the expected text in the training set to obtain phonetic characters; natural language processing on the expected text includes lexical analysis, syntactic analysis, semantic analysis, and context analysis of the expected text to obtain sentiment words, and convert the sentiment words into phonetic characters;

[0057] S3: The first convolutional neural network is used to extract features from Zhuyin characters, and the extracted features are converted into hidden vectors through the encoder; the hidden vectors are processed using a multi-head attention mechanism;

[0058] S4: Decode the hidden vector after the multi-head attention mechanism and input the decoded vector into the second convolutional neural network to obtain the speech acoustic features;

[0059] S5: Convert speech acoustic features into time domain waveforms and synthesize them into speech information;

[0060] S6: Mixing the speech information with the corresponding speech material, and performing preliminary feature extraction on the mixed speech information using symmetric Mel-frequency cepstral coefficients;

[0061] S7: Use a multi-scale deep convolutional neural network to perform deep feature extraction on the preliminary features to obtain multi-scale feature information;

[0062] S8: Combine multi-scale features to obtain detection results;

[0063] S9: Calculate the loss function of the model based on the detection results, continuously adjust the model parameters, and complete the model training when the loss function converges.

[0064] The process of preprocessing speech data includes: natural language sentiment analysis, pre-emphasis, framing and windowing of speech data; among them, natural language sentiment analysis includes lexical analysis, syntactic analysis, semantic analysis and context analysis of speech data, completing word segmentation processing and sentiment analysis to weight speech data containing emotional tone, so as to facilitate subsequent module processing; pre-emphasis is to multiply the speech data with a coefficient in the frequency domain, and the coefficient is positively correlated with the frequency of the speech data; framing processing includes segmenting the pre-emphasized speech data in combination with the results of natural language processing sentiment analysis to obtain short-time speech data; windowing processing includes multiplying the short-time speech data with a window function to obtain windowed short-time speech data.

[0065] The process of processing the hidden vector using the multi-head attention mechanism includes: the multi-head attention mechanism includes at least two sub-attention mechanisms, which aggregate the data processed by each word attention mechanism to obtain the hidden vector processed by the multi-head attention mechanism; the output of each sub-attention mechanism is:

[0066] head i =Attention(SW i S +FW i F +B)

[0067] The output of the multi-head attention mechanism is:

[0068] fhead=concat(head1,head2...head n )W

[0069] Among them, head i represents the output of the i-th sub-attention mechanism, Attention represents the attention mechanism, S and F are encoder inputs, B is the offset, and W i S 、W i Fare all sub-attention mechanism weight values, concat means connecting all sub-attention mechanism functions, and W is the weight value of each sub-attention mechanism.

[0070] The process of decoding the hidden vector after the multi-head attention mechanism includes: using the decoder to splice the output of the multi-head attention mechanism and the encoding vector output at the previous moment, and inputting the spliced ​​vector into the LSTM module to calculate the new encoding vector; the calculation formula of the splicing vector is:

[0071] S=S t +w s (F s *s t )+B s

[0072] Among them, S is the splicing vector, S t is the output of the multi-head attention mechanism, F s is the convolution operation, W s and B s are the parameters to be trained.

[0073] The process of using symmetric Mel-cepstral coefficients to perform preliminary feature extraction on the mixed speech information includes: constructing dense filters in the low-frequency and high-frequency frequency domains, generating a symmetric Mel-cepstral coefficient function based on the constructed filters, and performing preliminary feature extraction on the mixed speech information based on the constructed function; the expression of the symmetric Mel-cepstral coefficient function is:

[0074]

[0075] Among them, k is the frequency point number in Fourier transform, SG m (k) is the symmetric Mel cepstral function, G m (k) is the Mel cepstrum output, IG m (k) is the inverse Mel-frequency cepstrum output.

[0076] The process of using a multi-scale deep convolutional neural network to extract deep features from preliminary features includes: the output of each convolutional layer is used as the input of the next layer, and the convolution results of all convolutional layers are output to obtain multi-scale feature information; the convolution formula of each convolutional layer is:

[0077] S i =W i S i-1 +B i

[0078] Among them, S i is the output of the i-th convolutional layer, W i is the convolution kernel of the i-th layer, S i-1 is the output of the i-1th convolutional layer, Bi The offset is the parameter to be trained.

[0079] The speech detection model uses the CycleGAN model, which is equivalent to building two GAN networks to form a ring-shaped generative network to improve model performance. Therefore, the loss function consists of the adversarial loss of the two GAN networks and the cycle consistency loss between them. The adversarial loss is composed of XY and YX.

[0080] The first convolutional neural network loss function is:

[0081] L adv (X,Y)=E y～Pdata(y) [logD Y (y)]-E x～Pdata(x) [logDy(G(x))]

[0082] The loss function of the second convolutional neural network is:

[0083] L adv (Y,X)=E x～Pdata(x) [logD X (x)]-E y～Pdata(y) [logD X (F(y))]

[0084] The cycle consistency loss function is:

[0085] L C =E x～Pdata(x) [||F(G(x)-x)||]+E y～Pdata(y) [||G(F(y)-y)||]

[0086] The total loss function of the model is:

[0087] L t =L adv (X,Y)+L adv (Y,X)+L C

[0088] Wherein, E(*) represents the expected value, x represents the real speech input to the first convolutional neural network, y represents the real speech input to the second convolutional neural network, Pdata(x) represents the real speech distribution of the first convolutional neural network, Pdata(y) represents the real speech distribution of the second convolutional neural network, G(x) represents the generative network output of the first convolutional neural network, F(y) represents the generative network output of the second convolutional neural network, D(x) represents the discriminative network output of the first convolutional neural network, and D(y) represents the discriminative network output of the second convolutional neural network.

[0089] A synthetic speech detection system based on generative adversarial networks, such as Figure 1 As shown, it includes: speech preprocessor, generative adversarial network model trainer, and detection classifier.

[0090] The speech preprocessor can preprocess the input speech and process the speech segments into more ideal signals to facilitate the subsequent model trainer and detector work. Figure 3 As shown, the speech preprocessor mainly includes a natural language processing sentiment analysis module, a pre-emphasis module, a framing module, and a windowing module.

[0091] The natural language processing sentiment analysis module performs lexical analysis, syntactic analysis, semantic analysis and context analysis on the speech data, completes word segmentation and sentiment analysis, and weights the speech data containing emotional tone to facilitate subsequent module processing.

[0092] The pre-emphasis module eliminates the effects of the vocal cords and lips during speech production, compensating for the high-frequency components of the speech signal that are suppressed by the articulatory system. It also emphasizes high-frequency formants. This is achieved by multiplying the frequency domain by a coefficient that is positively correlated with frequency, thereby increasing the amplitude of high frequencies.

[0093] The framing module divides long speech signals into several short segments for processing. This is because speech signals are unstable at a macro level but stable at a micro level, with short-term stability. Subsequent processing requires the input signal to be stable, so the long signal is segmented.

[0094] The windowing module multiplies the speech signal by a window function to facilitate subsequent expansion. This allows for a more connected global representation and avoids the Gibbs effect. Furthermore, speech signals that are not originally periodic will also exhibit some of the characteristics of a periodic function.

[0095] like Figure 2 The generative adversarial network model trainer can perform speech synthesis and speech detection game learning to improve the effect of synthetic speech detection. The trained model will be introduced into the subsequent detection classifier to perform actual synthetic speech detection. Figure 4 As shown, the generative adversarial network model trainer includes: a speech generation module, a speech discrimination module, and a back propagation module.

[0096] The speech generation module can perform speech synthesis. Input the desired text and the target object speech material, and output the target object synthesized speech containing the desired text through deep network learning. Figure 5 As shown in the figure, the speech generation module mainly includes a text conversion unit, an encoding unit, a multi-head attention mechanism unit, a decoding unit, and a sound synthesis unit.

[0097] The text conversion unit uses natural language processing to convert long text paragraphs or sentences into more detailed word and character levels. Furthermore, it incorporates the pronunciation habits of a specific language and adds relevant identifiers to the intonation of speech to more closely resemble real-life speech. Ultimately, it converts long strings of text characters into phonetic characters.

[0098] The encoding unit is responsible for encoding the converted phonetic characters into latent vectors through several layers of convolutional networks.

[0099] The multi-head attention mechanism unit passes the latent vector into the multi-head attention mechanism to improve the synthesis effect and speed.

[0100] The decoding unit first recalculates the above code to obtain a new code through several layers of LSTM network, then obtains the output spectrum frame through the linear mapping layer, and finally obtains the speech acoustic characteristics through several layers of convolutional network.

[0101] The sound synthesis unit converts the above-mentioned speech acoustic features into time domain waveforms to synthesize the speech of a specific speaker.

[0102] The speech recognition module can perform the work of distinguishing and classifying the synthesized speech. The speech synthesized by the speech synthesis module is mixed with the real speech, and then sent to the speech recognition module for detection. It is expected that the recognition module can correctly classify the synthesized speech and the real speech. Figure 6 As shown in the figure, the speech recognition module mainly includes a feature extraction unit, a deep convolution unit, and a detection output unit.

[0103] The feature extraction unit first uses symmetric Mel-frequency cepstral coefficients to complete preliminary feature extraction, then performs further feature extraction through several layers of convolutional networks, and finally outputs the results to the deep convolution unit as the input of the unit.

[0104] The deep convolution unit constructs several layers of deep convolutional networks with different array convolution kernel sizes to further extract features, and then sends the features extracted by these groups of networks to the pooling layer to complete the pooling operation, and then splices these groups of pooling results to obtain higher-dimensional information.

[0105] The detection and output unit uses the fully connected layer to complete the discrimination and classification of the speech using the feature information obtained by the above splicing, classifies the input speech into one of the two categories of synthesized speech and non-synthesized speech, and outputs the result.

[0106] The back propagation module compares the output of the speech discrimination module with the correct result, constructs a loss function, and backpropagates it to the speech generation module and the speech discrimination module to optimize the network structure parameters within the above modules, thereby improving the synthesis quality of the generation module and the detection effect of the discrimination module.

[0107] The detection classifier introduces a trained generative adversarial network model and can perform actual synthetic speech detection, that is, the speech to be tested is input and sent to the detection classifier after preprocessing to output the detection result. Figure 7 As shown in FIG, the detection classifier mainly includes a speech detection module and a result classification module.

[0108] The speech detection module will introduce the trained generative adversarial network model to perform actual detection work.

[0109] The result classification module classifies and outputs the detection results obtained by the detection module, outputting the following for real speech: this speech is non-synthesized speech, and outputting the following for synthesized speech: this speech is synthesized speech, and displays the detected and analyzed sound spectrum to the user.

[0110] Another specific implementation of a synthetic speech detection system based on a generative adversarial network, such as Figure 8 As shown, it includes: speech preprocessor, generative adversarial network model trainer, and detection classifier.

[0111] The speech preprocessor can preprocess the input speech and process the speech segments into more ideal signals to facilitate the subsequent model trainer and detector. The speech preprocessor is mainly composed of the following modules: pre-emphasis module, framing module, and windowing module.

[0112] The pre-emphasis module eliminates the effects of the vocal cords and lips during speech production, compensating for the high-frequency components of the speech signal that are suppressed by the articulatory system. It also emphasizes high-frequency formants. This is achieved by multiplying the frequency domain by a coefficient that is positively correlated with frequency, thereby increasing the amplitude of high frequencies.

[0113] The framing module divides long speech signals into several short segments for processing. This is because speech signals are unstable at a macro level but stable at a micro level, with short-term stability. Subsequent processing requires the input signal to be stable, so the long signal is segmented.

[0114] The windowing module multiplies the speech signal by a window function to facilitate subsequent expansion. This allows for a more connected global representation and avoids the Gibbs effect. Furthermore, speech signals that are not originally periodic will also exhibit some of the characteristics of a periodic function.

[0115] The generative adversarial network model trainer can perform speech synthesis and speech detection game learning to improve the synthetic speech detection effect. The trained model will be introduced into the subsequent detection classifier to perform actual synthetic speech detection work, and can accept reverse feedback from the detection classifier to continuously adjust parameters and optimize the model according to the actual detection effect. Figure 9As shown in the figure, the generative adversarial network model trainer mainly consists of a speech generation module, a speech discrimination module, a back propagation module, and a model optimization module.

[0116] The data processed by the speech generation module, speech discrimination module, and back propagation module are the same as those processed by the speech generation module, speech discrimination module, and back propagation module in specific implementation mode 1.

[0117] The model optimization module receives the detection result feedback with labels (synthetic and non-synthetic) from the detection classifier and constructs a loss function to adjust the network structure parameters and number of network layers of the training model to continuously improve the model detection effect.

[0118] The detection classifier introduces the trained generative adversarial network model and can perform actual synthetic speech detection, that is, the speech to be tested is input, and after preprocessing, it is sent to the detection classifier to output the detection result, and the detection result is labeled (synthetic or non-synthetic) and sent back to the generative adversarial network model trainer. Figure 10 As shown in the figure, the detection classifier mainly includes a speech detection module, a result classification module, and a result labeling module.

[0119] The speech detection module will introduce the trained generative adversarial network model to perform actual detection work.

[0120] The result classification module classifies and outputs the detection results obtained by the above detection module. For real speech, it outputs: this speech is non-synthesized speech; for synthesized speech, it outputs: this speech is synthesized speech, and displays the detected and analyzed sound spectrum to the user.

[0121] The result labeling module labels the detection results and actual classifications (synthetic and non-synthetic) respectively and returns them to the generative adversarial network model trainer, and performs reverse supervision optimization of the model by constructing a loss function and other methods.

[0122] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A synthetic speech detection method based on generative adversarial network, characterized in that: include: Acquire the voice data to be detected and preprocess the voice data; Use the trained speech detection model to detect the preprocessed speech data, obtain the detection results, and classify and mark the detection results; The speech detection model is a generative adversarial network; The process of training a speech detection model includes: S1: Obtain a training set, where the data includes expected text and voice materials; S2: Perform natural language sentiment analysis on the expected text in the training set to obtain phonetic characters; the natural language sentiment analysis includes performing lexical analysis, syntactic analysis, semantic analysis, and context analysis on the expected text to obtain sentiment words, and convert the sentiment words into phonetic characters; S3: The first convolutional neural network is used to extract features from Zhuyin characters, and the extracted features are converted into hidden vectors through the encoder; the hidden vectors are processed using a multi-head attention mechanism; The process of processing the hidden vector using the multi-head attention mechanism includes: the multi-head attention mechanism includes at least two sub-attention mechanisms, which aggregate the data processed by each word attention mechanism to obtain the hidden vector processed by the multi-head attention mechanism; the output of each sub-attention mechanism is: head i =Attention(SW i S +FW i F +B) The output of the multi-head attention mechanism is: fhead=concat(head1,head2...head n )W Among them, head i represents the output of the i-th sub-attention mechanism, Attention represents the attention mechanism, S and F are encoder inputs, B is the offset, and W i S 、W i F are all sub-attention mechanism weights, concat means connecting all sub-attention mechanism functions, and W is the weight of each sub-attention mechanism; S4: Decode the hidden vector after the multi-head attention mechanism and input the decoded vector into the second convolutional neural network to obtain the speech acoustic features; The process of decoding the hidden vector after the multi-head attention mechanism includes: using the decoder to splice the output of the multi-head attention mechanism and the encoding vector output at the previous moment, and inputting the spliced ​​vector into the LSTM module to calculate the new encoding vector; the calculation formula of the splicing vector is: S=S t +w s (F s *s t )+B s Among them, S is the splicing vector, S t is the output of the multi-head attention mechanism, F s is the convolution operation, W s and B s are the parameters to be trained; S5: Convert speech acoustic features into time domain waveforms and synthesize them into speech information; S6: Mixing the speech information with the corresponding speech material, and performing preliminary feature extraction on the mixed speech information using symmetric Mel-frequency cepstral coefficients; S7: Use a multi-scale deep convolutional neural network to perform deep feature extraction on the preliminary features to obtain multi-scale feature information; S8: Combine multi-scale features to obtain detection results; S9: Calculate the loss function of the model based on the detection results, continuously adjust the model parameters, and complete the model training when the loss function converges.

2. A synthetic speech detection method based on a generative adversarial network according to claim 1, characterized in that: The process of preprocessing speech data includes: natural language sentiment analysis, pre-emphasis, framing and windowing processing of speech data; pre-emphasis is to multiply the speech data with a coefficient in the frequency domain, and the coefficient is positively correlated with the frequency of the speech data; framing processing includes segmenting the pre-emphasized speech data in combination with the results of natural language processing sentiment analysis to obtain short-time speech data; windowing processing includes multiplying the short-time speech data with a window function to obtain windowed short-time speech data.

3. The synthetic speech detection method based on generative adversarial network according to claim 1, characterized in that: The process of using symmetric Mel-cepstral coefficients to perform preliminary feature extraction on the mixed speech information includes: constructing dense filters in the low-frequency and high-frequency frequency domains, generating a symmetric Mel-cepstral coefficient function based on the constructed filters, and performing preliminary feature extraction on the mixed speech information based on the constructed function; the expression of the symmetric Mel-cepstral coefficient function is: Among them, k is the frequency point number in Fourier transform, SG m (k) is the symmetric Mel cepstral function, G m (k) is the Mel cepstrum output, IG m (k) is the inverse Mel-frequency cepstrum output.

4. The synthetic speech detection method based on generative adversarial network according to claim 1, characterized in that: The process of using a multi-scale deep convolutional neural network to extract deep features from preliminary features includes: the output of each convolutional layer is used as the input of the next layer, and the convolution results of all convolutional layers are output to obtain multi-scale feature information; the convolution formula of each convolutional layer is: S i =W i S i-1 +B i Among them, S i is the output of the i-th convolutional layer, W i is the convolution kernel of the i-th layer, S i-1 is the output of the i-1th convolutional layer, B i The offset is the parameter to be trained.

5. The synthetic speech detection method based on generative adversarial network according to claim 1, characterized in that: The loss function of the model includes the first convolutional neural network loss function, the second convolutional neural network loss function, and the cycle consistency loss function; The first convolutional neural network loss function is: L adv (X,Y)=E y～Pdata(y) [logD Y (y)]-E x～Pdata(x) [logDy(G(x))] The loss function of the second convolutional neural network is: L adv (Y,X)=E x～Pdata(x) [logD X (x)]-E y～Pdata(y) [logD X (F(y))] The cycle consistency loss function is: L C =E x～Pdata(x) [||F(G(x)-x)||]+E y～Pdata(y) [||G(F(y)-y)||] The total loss function of the model is: L t =L adv (X,Y)+L adv (Y,X)+L C Wherein, E(*) represents the expected value, x represents the real speech input to the first convolutional neural network, y represents the real speech input to the second convolutional neural network, Pdata(x) represents the real speech distribution of the first convolutional neural network, Pdata(y) represents the real speech distribution of the second convolutional neural network, G(x) represents the generative network output of the first convolutional neural network, F(y) represents the generative network output of the second convolutional neural network, D(x) represents the discriminative network output of the first convolutional neural network, and D(y) represents the discriminative network output of the second convolutional neural network.

Images