Forged voice detection method, device, storage medium and electronic device

Abstract

The present application discloses a method, device, storage medium and electronic device for detecting forged speech. The method comprises: collecting a speech signal to be detected, converting the speech signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete speech signal; performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing quadratic interpolation on the amplitude spectrum to obtain the peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal; extracting the linear frequency cepstral coefficients of the discrete speech signal, splicing the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature; and inputting the fusion feature into a trained forged speech detection model to perform forged speech recognition. The present application can improve the accuracy of forged speech detection.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things security technology, and in particular to a forged voice detection method, device, storage medium and electronic device. Background Art

[0002] Voice has become an important way for human-computer interaction in smart IoT (such as smart home) and smart car networking. IoT devices can be authenticated through human voice. However, with the development of artificial intelligence technology, forged voices using different algorithms have brought great challenges to the development of smart IoT voice devices. Attackers can imitate and forge the voice of real speakers through various means such as voice playback, speech synthesis and text conversion, deceive IoT devices and obtain user permissions. Therefore, a forged voice detection method is urgently needed to improve the reliability and security of IoT device authentication and ensure the stable operation of IoT devices.

[0003] At present, the existing related forged voice detection technology is mainly detected through spectrum analysis. However, due to the complexity of the characteristic information carried by the voice signal itself, the distinction between real voice and forged voice often depends on some subtle but key features. These features are often mixed together and difficult to extract and distinguish. Therefore, the detection accuracy of the existing related forged voice detection technology is not high. Summary of the invention

[0004] The embodiments of the present application provide a forged voice detection method, device, storage medium and electronic device, which can improve the accuracy of forged voice detection technology.

[0005] The present application provides a method for detecting forged voice, comprising:

[0006] Collecting a voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal;

[0007] Performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal;

[0008] Extracting linear frequency cepstral coefficients of the discrete speech signal, and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature;

[0009] The fused features are input into a trained forged speech detection model to perform forged speech recognition.

[0010] Furthermore, the forged voice detection method, wherein the step of collecting the voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal, comprises:

[0011] Collecting a voice signal to be detected, and sampling the voice signal at a preset sampling frequency to obtain the time series digital signal;

[0012] The time series digital signal is framed, and the framed time series digital signal is windowed to obtain the discrete speech signal.

[0013] Furthermore, the forged voice detection method, wherein the Fourier transform is performed on the discrete voice signal to obtain an amplitude spectrum, a secondary interpolation is performed on the amplitude spectrum to obtain a peak value of each frequency window, and the peak values ​​are combined to obtain a peak-frequency matrix of the discrete voice signal, comprises:

[0014] Performing a Fourier transform operation on the discrete speech signal to extract frequency domain information of the speech signal;

[0015] Taking the modulus of the complex function of the frequency domain information in the frequency domain to obtain an amplitude spectrum;

[0016] Fitting the frequency window and amplitude of each frame on the amplitude spectrum to determine a parabola, determining the frequency window corresponding to the amplitude peak by assuming the positions of three known amplitude values ​​and performing quadratic interpolation, repeating the quadratic interpolation multiple times to obtain multiple peak frequencies;

[0017] A plurality of the peak frequencies are concatenated to obtain a peak frequency matrix.

[0018] Furthermore, in the forged speech detection method, the step of extracting the linear frequency cepstral coefficients of the discrete speech signal and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature comprises:

[0019] Taking the modulus square of the complex function of the discrete speech signal in the frequency domain to obtain a power energy spectrum of the discrete speech signal;

[0020] Extracting frequency band features of the power energy spectrum through a linear scale filter and performing discrete cosine transform to obtain linear frequency cepstrum coefficients;

[0021] The peak frequency and the linear frequency cepstrum coefficient are spliced ​​according to the number of frames to obtain the fusion feature.

[0022] Furthermore, in the forged voice detection method, the forged voice detection model includes a two-dimensional convolutional layer, a Res2Net module, a global average pooling layer, a fully connected layer and a softmax classification module;

[0023] The step of inputting the fusion features into a trained forged speech detection model for forged speech recognition includes:

[0024] Inputting the fused features into the two-dimensional convolutional layer to perform a convolution operation to extract local features;

[0025] Input the local features into the Res2Net module to obtain a residual matrix;

[0026] Input the residual matrix into the global average pooling layer to generate a pooled eigenvalue corresponding to each channel, and realize a linear combination of the previous layer through a fully connected layer to obtain a one-dimensional eigenvector;

[0027] The one-dimensional feature vector is input into a softmax classification module to obtain a forged speech recognition result.

[0028] Furthermore, the forged speech detection method, wherein the frequency window and amplitude of each frame on the amplitude spectrum are fitted to determine a parabola, the frequency window corresponding to the amplitude peak is determined by assuming the positions of three known amplitude values ​​for secondary interpolation, and the secondary interpolation is repeated multiple times to obtain multiple peak frequencies, including:

[0029] Fitting the frequency window and amplitude of each frame on the amplitude spectrum determines a parabola:

[0030]

[0031] in, is the independent variable of the parabola, i.e. the frequency window, is the dependent variable of the parabola, i.e., the amplitude value, , are the parameters of the parabola;

[0032] Assume that the positions of the three known amplitude values ​​are interpolated quadratically to determine the frequency window corresponding to the amplitude peak. Assume , , are three known amplitude values, and:

[0033]

[0034] By substituting the positions of the three amplitude values ​​into the fitted parabola, we can obtain:

[0035]

[0036] Calculate the frequency window corresponding to the amplitude peak:

[0037]

[0038] For the magnitude spectrum Frame, repeat the secondary interpolation multiple times to get multiple peak frequencies Among them, the peak frequency The calculation formula is:

[0039]

[0040] in, is the peak frequency, is the frequency window corresponding to the peak amplitude of the last quadratic interpolation, is the frequency window corresponding to the amplitude peak, is the speech sampling rate, is the window length.

[0041] Furthermore, in the forged voice detection method, the training process of the forged voice detection model includes:

[0042] Acquire a training set and a test set, wherein the training set includes a speech sample signal, and the test set includes a speech test signal;

[0043] Extracting the peak frequencies of the speech sample signal and the speech test signal, extracting the linear frequency cepstral coefficients of the speech sample signal and the speech test signal, and concatenating the peak frequencies and the linear frequency cepstral coefficients to obtain a training fusion vector and a test fusion vector;

[0044] Inputting the training fusion vector into the forged speech detection model to obtain a prediction result;

[0045] According to the prediction result and the true value label of the voice sample signal, a binary cross entropy loss function is calculated, a gradient is calculated by back propagation, and the weights and biases of the parameters in the forged voice detection model are adjusted multiple times by iteration;

[0046] The test fusion vector is input into the forged speech detection model to determine whether the prediction accuracy of the forged speech detection model reaches a preset value. If so, the training is terminated.

[0047] The embodiment of the present application also provides a forged voice detection device, comprising:

[0048] The acquisition and preprocessing module is used to acquire the voice signal to be detected, convert the voice signal into a time-series digital signal, and preprocess the time-series digital signal to obtain a discrete voice signal;

[0049] A first processing module is used to perform Fourier transform on the discrete speech signal to obtain an amplitude spectrum, perform secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combine the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal;

[0050] A second processing module is used to extract the linear frequency cepstral coefficients of the discrete speech signal, and concatenate the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature;

[0051] The recognition module is used to input the fusion features into a trained forged speech detection model to perform forged speech recognition.

[0052] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute any of the forged voice detection methods described above.

[0053] An embodiment of the present application further provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to perform the steps in any one of the forged voice detection methods described above.

[0054] The forged voice detection method, device, storage medium and electronic device provided by the present application extract the peak frequency and linear frequency cepstrum coefficient of the voice signal, fuse the two to obtain a fusion feature, and identify the forged voice based on the fusion feature. The peak frequency represents the change information of the voice frequency. The present application improves the accuracy of forged voice detection by increasing the attention to the peak change information of the key frequency of the voice, accurately locating the difference between the forged voice and the real voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0056] Figure 1 This is a flowchart of a forged voice detection method provided in an embodiment of the present application.

[0057] Figure 2 This is a visual scatter plot of the classification effect of the forged voice detection method of an embodiment of the present application on the ASVspoof 2019 validation set.

[0058] Figure 3 This is one of the example diagrams of the visualization results of the detection effect of the method provided in the embodiment of the present application on the ASVspoof 2019 dataset for various types of attacks.

[0059] Figure 4 The second example diagram of the visualization results of the detection effect of the method provided in the embodiment of the present application on various types of attacks on the ASVspoof 2019 dataset.

[0060] Figure 5A schematic diagram of the structure of a forged voice detection device provided in an embodiment of the present application.

[0061] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0062] Figure 7 Another schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0064] At present, the shortcomings of existing forged voice detection technologies are mainly reflected in three aspects. First, the characteristic information carried by the voice signal itself is complex. The distinction between real voice and forged voice often depends on some subtle but key features, which are often mixed together and difficult to extract and distinguish; second, the types and methods of forged voice are increasing day by day, and advanced synthesis algorithms have continuously improved the realism of voice. Training models with known forgery methods to detect more advanced synthesis algorithms has become an important evaluation indicator of model generalization; third, it is necessary to balance user experience and device security. Multi-factor detection methods can improve the security of identity authentication, but users are required to purchase professional equipment or cooperate with specific operations, which reduces the user experience. While ensuring the accuracy of detection, the requirements for users should be reduced as much as possible to improve the user experience.

[0065] To solve the above problems, the embodiments of the present application provide a forged voice detection method, device, storage medium and electronic device. The forged voice detection device provided in the embodiments of the present application can be integrated in an electronic device, which can be a terminal, a server and other devices, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0066] The following first explains and illustrates the technical terms of this application:

[0067] Fourier transform: A linear integral transform that expresses a function that satisfies certain conditions as a linear combination of trigonometric functions or their integrals. It is used to transform signals between the time domain and the frequency domain or between the space domain and the frequency domain, and has a wide range of applications in physics, engineering, signal processing, image processing, and other fields. The properties of Fourier transform include linearity, time shift, frequency shift, scale transformation, convolution, etc. Its variants include continuous Fourier transform, discrete Fourier transform, etc.

[0068] Linear frequency cepstrum coefficients: A technique used for speech analysis and coding that makes the characteristics of speech signals more prominent and easier to analyze and process by converting the spectrum into cepstrum form. This technique obtains coefficients by performing discrete Fourier transform or discrete cosine transform on the cepstrum and is widely used in speech compression and transmission, helping to reduce the amount of data while maintaining the sound quality and intelligibility of speech signals.

[0069] Res2Net: A deep convolutional neural network architecture that aims to address the problems of traditional ResNet, such as insufficient modeling of features of different scales and resolutions and limited network depth.

[0070] Global max pooling: A pooling operation that extracts the maximum value of each channel of the input tensor to form a new tensor. Global max pooling is often used in the last layer of a convolutional neural network to convert the output of the convolutional layer into a fixed-size vector for classification or regression tasks. Through this operation, the maximum activation value of each channel is retained as the global feature representation of the channel.

[0071] Softmax: A function that converts the output of multiple classifications into a probability distribution. It can convert any real-valued score into a probability distribution, and the sum of the probabilities is 1.

[0072] ASVspoof: A program focused on deception and adversarial research in automatic speaker verification systems, providing standard datasets and evaluation platforms to support researchers in developing and testing countermeasures against spoofing attacks.

[0073] EER: A metric used to measure the performance of a classifier. EER is a common value when the false acceptance rate and false rejection rate are equal, and is used to indicate the accuracy of a biometric system. The lower the EER value, the more accurate the system.

[0074] t-DCF: A key metric used to evaluate system performance. t-DCF takes into account the weights of the false acceptance rate and the false rejection rate and eliminates the effect of different speech sample lengths by normalizing them to the same duration. The lower the t-DCF value, the higher the accuracy of the system.

[0075] See also Figure 1 , Figure 1The flowchart of the forged voice detection method provided in the embodiment of the present application is applied in an electronic device. The forged voice detection method includes the following steps:

[0076] S1, collecting the voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal.

[0077] In one embodiment, step S1 includes the following steps:

[0078] S11, collecting a voice signal to be detected, and sampling the voice signal at a preset sampling frequency to obtain a time-series digital signal.

[0079] The voice signal to be detected input by the user collected by the speaker of the IoT device is collected, and an amplitude sample of the signal is extracted at a certain time interval (i.e., sampling period) from the continuous analog signal on the time axis to form a discrete time series digital signal.

[0080] S12, dividing the time series digital signal into frames, and windowing the framed time series digital signal to obtain a discrete speech signal.

[0081] In order to utilize the short-term stability of speech signals, the discrete time-series digital signals are divided into frames. A voice, the number of frames corresponding to the entire voice signal It can be expressed as:

[0082]

[0083] in, is the number of frames, is the length of the speech signal, is the frame length, It is frame shift, overlapping part The frame length is subtracted from the frame shift. When the time-series digital signal is framed, the frame length is 320ms and the frame shift is 160ms.

[0084] In order to avoid the truncation problem caused by different speech lengths during framing, the speech sequence after framing is windowed to obtain a discrete speech signal. The general formula of the Hamming window can be expressed as:

[0085]

[0086] in, is the value of the window function in the discrete time series, is the index in the frame sequence, is the window length.

[0087] S2, perform Fourier transform on the discrete speech signal to obtain the amplitude spectrum, perform secondary interpolation on the amplitude spectrum to obtain the peak value of each frequency window, and combine the peak values ​​to obtain the peak-frequency matrix of the discrete speech signal.

[0088] In one embodiment, step S2 comprises the following steps:

[0089] S21, performing Fourier transform operation on the discrete speech signal to extract frequency domain information of the speech signal.

[0090] The specific calculation can be done by the following formula:

[0091]

[0092] in, Represents the complex function of the digital speech signal in the frequency domain, is a frequency variable, which represents the frequency of the digital speech signal in the frequency domain. is the function of the digital speech signal in the time domain, is the index in the frame sequence, is the window length, is the total number of sampling points in the frequency domain, which is the same as N.

[0093] S22, taking the modulus of the complex function of the frequency domain information in the frequency domain to obtain an amplitude spectrum.

[0094] Complex function of discrete speech signal in frequency domain Modulo is performed to obtain the amplitude spectrum of the speech signal. The horizontal axis of the amplitude spectrum represents the time frame, and the vertical axis represents the frequency bin (i.e. frequency window). Each frame and bin corresponds to a value, which represents the amplitude.

[0095] S23, fitting the frequency window and amplitude of each frame on the amplitude spectrum to determine a parabola, assuming the positions of three known amplitude values ​​to perform quadratic interpolation to determine the frequency window corresponding to the amplitude peak, repeating the quadratic interpolation multiple times to obtain multiple peak frequencies.

[0096] Specifically, step S23 includes:

[0097] S231, fitting the frequency window and amplitude of each frame on the amplitude spectrum to determine a parabola:

[0098]

[0099] in, is the independent variable of the parabola, i.e. the frequency window, is the dependent variable of the parabola, i.e., the amplitude value, , are the parameters of the parabola;

[0100] S232, assuming that the positions of the three known amplitude values ​​are interpolated twice to determine the frequency window corresponding to the amplitude peak, assuming that , , are three known amplitude values, and:

[0101]

[0102] S233, bring the positions of the three amplitude values ​​into the fitted parabola, and we can get:

[0103]

[0104] S234, calculate the frequency window corresponding to the amplitude peak:

[0105]

[0106] S235, for the magnitude spectrum Frame, repeat the secondary interpolation multiple times to get multiple peak frequencies Among them, the peak frequency The calculation formula is:

[0107]

[0108] in, is the peak frequency, is the frequency window corresponding to the peak amplitude of the last quadratic interpolation, is the frequency window corresponding to the amplitude peak, is the speech sampling rate, is the window length. In this embodiment, the number of repetitions of the secondary interpolation is 16 times, and the speech sampling rate of the secondary interpolation is 16 KHz.

[0109] S24, splicing The peak frequency of each frame signal is obtained to obtain the peak frequency matrix.

[0110] S3, extract the linear frequency cepstral coefficients of the discrete speech signal, and concatenate the peak frequency and the linear frequency cepstral coefficients to obtain the fusion feature.

[0111] In one embodiment, step S3 includes the following steps:

[0112] S31, taking the modulus square of the complex function of the discrete speech signal in the frequency domain to obtain the power energy spectrum of the discrete speech signal.

[0113] It can be calculated by the following formula:

[0114]

[0115] in, is the power energy spectrum, is the complex function of the discrete speech signal in the frequency domain, is the window length.

[0116] S32, extracting frequency band features of the power energy spectrum through a linear scale filter, and performing discrete cosine transform to obtain linear frequency cepstrum coefficients.

[0117] Specifically, a set of linear scale filters is used to extract the frequency band features of the power energy spectrum, and a discrete cosine transform is performed to obtain the frequency cepstrum coefficients, and only 13 of the frequency cepstrum coefficients, namely the linear frequency cepstrum coefficients, are retained. The formula for discrete cosine transform is:

[0118]

[0119] in, Represents discrete speech signals Order linear frequency cepstrum coefficients, represents the order of the linear frequency cepstrum coefficients, represents the frequency band characteristics of the short time frame, is the index in the frame sequence, is the window length, is the number of linear scaling filters.

[0120] S33, splicing the peak frequency and the linear frequency cepstrum coefficient according to the frame number to obtain a fusion feature.

[0121] S4, inputting the fused features into the trained forged speech detection model to perform forged speech recognition.

[0122] Specifically, the fake speech detection model includes a two-dimensional convolutional layer, a Res2Net module, a global average pooling layer, a fully connected layer, and a softmax classification module (specifically, a softmax classification function);

[0123] Step S4 includes the following steps:

[0124] S41, input the fused features into the two-dimensional convolutional layer for convolution operation to extract local features.

[0125] S42, input the local features into the Res2Net module to obtain the residual matrix.

[0126] Specifically, the Res2Net module is used to group the feature information extracted by the previous layer to form multiple sub-matrices. These sub-matrices maintain the same height and width, but may have different numbers of channels. Then, convolution operations are performed on each sub-matrix to capture its unique features through a carefully designed convolution layer network. These convolution layers not only have different depths, but also different convolution kernel sizes, aiming to capture feature information at different scales. Inside the sub-matrix, a residual connection mechanism is introduced to effectively transfer feature information. The original input features of the sub-matrix are added to the features obtained after the convolution operation to form residual features. The residual features are then passed to the convolution operation of the next layer or other subsequent processing steps. The calculation formula is expressed as:

[0127]

[0128] in, Representative The output of the convolution operation of each group. Except for the first group, each group of sub-matrices performs a convolution operation with a convolution kernel size of 3, which is recorded as For groups other than the first and last group, the output of the current group is first added to the output of the previous group, and the result is the result of the residual connection. Finally, the outputs of all sub-matrices are concatenated in the channel dimension, and the number of channels of the final feature is adjusted through additional convolutional layers.

[0129] The feature information extracted from the previous level is input into multiple Res2Net modules in sequence to obtain the residual matrix.

[0130] S43, input the residual matrix into the global average pooling layer, generate the pooled eigenvalue corresponding to each channel, realize the linear combination of the previous layer through the fully connected layer, and obtain a one-dimensional feature vector.

[0131] S44, input the one-dimensional feature vector into the softmax classification module to obtain the forged speech recognition result.

[0132] Before step S4, the forged speech detection model needs to be trained. The training process is as follows:

[0133] SA1, build a fake speech detection model. The fake speech detection model includes three two-dimensional convolutions with a kernel size of 3 and a stride of 1, 16 Res2Net blocks, a global average pooling layer, a fully connected layer and a softmax classification function.

[0134] SA2 uses the training set data of the public dataset ASVspoof 2019 to train the forged speech detection model to achieve the highest detection accuracy, and uses the test set for verification to obtain the pre-trained model parameters.

[0135] Step SA2 specifically includes:

[0136] SA21, load the speech sample signals in the ASVspoof 2019 speech training set and the speech test signals in the test set, extract the peak frequencies of all speech sample signals and all speech test signals, extract the linear frequency cepstrum coefficients of all speech sample signals and all speech test signals, and splice them into fusion vectors respectively;

[0137] SA22, convolves the fusion vector of the speech sample signal through three two-dimensional convolutional layers to extract the local features of the speech sample signal and capture feature information at different levels;

[0138] SA23, the feature information extracted from the previous level is input into multiple Res2Net modules in sequence to obtain the residual matrix ;

[0139] SA24, the residual matrix The global average pooling layer is input to generate the converged feature value corresponding to each channel. The linear combination of the previous layer is realized through the fully connected layer to obtain a one-dimensional feature vector. The one-dimensional feature vector is input into the softmax function to obtain the prediction result. ;

[0140] SA25, according to the prediction results and the labels of the speech sample signals , use the binary cross entropy loss function to back propagate and calculate the gradient, and adjust the weights and biases of the model. The expression of the binary cross entropy loss function is:

[0141]

[0142] in, represents the binary cross entropy loss function, where is the actual label value, is the prediction result of the model, indicating the probability of belonging to the positive category;

[0143] SA26, using the voice test signal to detect whether the model accuracy has increased, if so, proceed to the next training, otherwise terminate the training and output the pre-trained model parameters. Alternatively, determine whether the prediction accuracy of the forged voice detection model has reached a preset value, if so, terminate the training.

[0144] SA3, load the pre-trained model parameters and wait for user voice input.

[0145] In one embodiment, after step S4, the following may be included:

[0146] S51, deploying the trained forged voice detection model in IoT devices;

[0147] S52, inputting the fusion features of the user input voice into the forged voice detection model to obtain a prediction result, and judging whether the voice signal to be tested is a forged voice according to the prediction result.

[0148] For the first difficulty, most of the current forged voice detections do not pay enough attention to the voice itself, and lack the process of extracting key voice information, which is the key to distinguishing real voice from forged voice. The present invention extracts the change information of the frequency peak through secondary interpolation, improves the attention to the key information of the voice, and enhances the accuracy of the model. For the second difficulty, most of the existing detection methods are aimed at a certain type of forged voice, and cannot accurately detect various types of forged voices in the real environment. The peak frequency proposed in this application has a good ability to distinguish various types of forged voices. At the same time, Res2Net is used as the basic framework of the model, which is conducive to improving the feature receptive field of the model and extracting multi-scale voice features. For the third difficulty, some existing multi-factor detection methods require users to purchase special equipment and require users to cooperate with specific postures, which cannot balance user experience and authentication security. The present invention detects forged voices based on the characteristics of the voice itself rather than additional physical quantities, such as acceleration, posture, and facial expressions, which greatly improves the user experience.

[0149] In general, compared with the prior art, this application has the following beneficial effects:

[0150] High detection accuracy: The present invention innovatively proposes a new acoustic feature for distinguishing forged speech, namely peak frequency. By extracting the change information of speech frequency, the focus on the change information of the peak value of key speech frequency is increased, and the difference between forged speech and real speech is more accurately located, thus improving the accuracy of forged speech detection.

[0151] Strong versatility: On the one hand, peak frequency indicates the change in speech frequency, and various types of forged speech have certain differences in the frequency domain compared to real speech. On the other hand, the forged speech detection model uses Res2Net blocks to extract information. Res2Net increases the receptive field of features through grouped convolution operations to avoid overfitting. It has been verified that this invention has excellent detection accuracy for various types of forged speech;

[0152] Excellent user experience: The application can be easily integrated into IoT devices, including smart home devices, wearable devices, industrial IoT devices, etc. Users do not need to purchase dedicated equipment or maintain a specific posture, and it can be deployed without the user feeling anything, which greatly optimizes the user experience.

[0153] Figure 2This is a visual scatter plot of the classification effect of the forged voice detection method of an embodiment of the present application on the ASVspoof 2019 validation set. The indicator is EER (equal error rate). In the figure, circles represent real voice samples True (real voice), and × represents forged voice samples Forged (deceptive voice). The samples show significant clustering phenomenon in the figure, proving that the forged voice detection method provided by the present application can accurately distinguish between forged voice and real voice.

[0154] Figure 3 One of the example diagrams of the visualization results of the detection effect of the method provided in the embodiment of the present application on the ASVspoof 2019 dataset for various types of attacks, Figure 4 The second example diagram of the visualization results of the detection effect of the method provided in the embodiment of the present application on various types of attacks on the ASVspoof 2019 dataset, with the indicators being EER and t-DCF respectively. In the figure, A01-A19 use different algorithms. The following describes in detail the generation algorithms of various types of fake speech, where TTS represents text-to-speech algorithm, VC represents speech-to-speech algorithm, A01 represents TTS neural waveform model, A02 represents TTS vocoder model, A03 represents TTS vocoder model, A04 represents TTS waveform connection, A05 represents VC vocoder model, A06 represents VC spectral filtering, A07 represents TTS vocoder model + GAN, A08 represents TTS neural waveform model, A09 represents TTS vocoder model, A10 represents TTS neural waveform model, A11 represents TTS Griffin_lim algorithm, A12 represents TTS neural waveform model, A13 represents TTS_VC waveform concatenation + waveform filtering, A14 represents TTS_VC vocoder model, A15 represents TTS_VC neural waveform model, A16 represents TTS waveform concatenation, A17 represents VC waveform filtering, and A18 represents VC Vocoder model, A19 represents VC spectrum filtering. The method of the present application has excellent detection capability compared with many other methods, which further proves the generalization capability of the forged speech detection method of the present application.

[0155] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a forged voice detection device. The forged voice detection device can be implemented as an independent entity or integrated in an electronic device. The electronic device can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0156] See also Figure 5 , Figure 5The forged voice detection device provided by the embodiment of the present application is specifically described, and is applied to an electronic device. The forged voice detection device may include:

[0157] The acquisition and preprocessing module is used to acquire the voice signal to be detected, convert the voice signal into a time-series digital signal, and preprocess the time-series digital signal to obtain a discrete voice signal;

[0158] The first processing module is used to perform Fourier transform on the discrete speech signal to obtain an amplitude spectrum, perform secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combine the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal;

[0159] The second processing module is used to extract the linear frequency cepstral coefficients of the discrete speech signal, and concatenate the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature;

[0160] The recognition module is used to input the fused features into the trained forged speech detection model to perform forged speech recognition.

[0161] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0162] In addition, an embodiment of the present application also provides an electronic device, which may be a computer, a tablet computer, or other device. Figure 6 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0163] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or loading applications stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device as a whole.

[0164] In this embodiment, the processor 401 in the electronic device 400 loads instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 runs the application programs stored in the memory 402 to implement various functions:

[0165] Collecting a voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal;

[0166] Performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal;

[0167] Extracting linear frequency cepstral coefficients of the discrete speech signal, and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature;

[0168] The fused features are input into a trained forged speech detection model to perform forged speech recognition.

[0169] The electronic device can implement the steps in any embodiment of the forged voice detection method provided in the embodiments of the present application, and therefore can achieve the beneficial effects that can be achieved by any forged voice detection method provided in the embodiments of the present invention. For details, please refer to the previous embodiments and will not be repeated here.

[0170] Figure 7 The specific structural block diagram of the electronic device provided by the embodiment of the present invention is shown, and the electronic device can be used to implement the forged voice detection method provided in the above embodiment. The electronic device 500 can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a notebook computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0171] The RF circuit 510 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks may use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0172] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizing the functions of taking pictures with the front camera, processing the captured images, and switching the display color of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely arranged relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0173] The input unit 530 may be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function control.

[0174] The display unit 540 may be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541, which may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.

[0175] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another terminal through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earplug jack to provide communication between an external headset and the electronic device 500.

[0176] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it is not a necessary component of the electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0177] The processor 580 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580.

[0178] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 590 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0179] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors. One or more programs include instructions for performing the following operations:

[0180] Collecting a voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal;

[0181] Performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal;

[0182] Extracting linear frequency cepstral coefficients of the discrete speech signal, and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature;

[0183] The fused features are input into a trained forged speech detection model to perform forged speech recognition.

[0184] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0185] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores a plurality of instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the forged voice detection method provided in the embodiment of the present invention.

[0186] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0187] Since the instructions stored in the storage medium can execute the steps in any embodiment of the forged voice detection method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any forged voice detection method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details, which will not be repeated here.

[0188] The forged voice detection method, device, storage medium and electronic device provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the present application are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technicians in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting forged speech, characterized in that: The method comprises: Collecting a voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal; Performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal; Extracting linear frequency cepstral coefficients of the discrete speech signal, and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature; The fused features are input into a trained forged speech detection model to perform forged speech recognition.

2. The forged voice detection method according to claim 1, characterized in that: The method of collecting a voice signal to be detected, converting the voice signal into a time-series digital signal, and preprocessing the time-series digital signal to obtain a discrete voice signal includes: Collecting a voice signal to be detected, and sampling the voice signal at a preset sampling frequency to obtain the time series digital signal; The time series digital signal is framed, and the framed time series digital signal is windowed to obtain the discrete speech signal.

3. The forged voice detection method according to claim 1, characterized in that: The method of performing Fourier transform on the discrete speech signal to obtain an amplitude spectrum, performing secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combining the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal comprises: Performing a Fourier transform operation on the discrete speech signal to extract frequency domain information of the speech signal; Taking the modulus of the complex function of the frequency domain information in the frequency domain to obtain an amplitude spectrum; Fitting the frequency window and amplitude of each frame on the amplitude spectrum to determine a parabola, performing quadratic interpolation by assuming the positions of three known amplitude values ​​to determine the frequency window corresponding to the amplitude peak, repeating the quadratic interpolation multiple times to obtain multiple peak frequencies; A plurality of the peak frequencies are concatenated to obtain a peak frequency matrix.

4. The forged voice detection method according to claim 1, characterized in that: The extracting the linear frequency cepstral coefficients of the discrete speech signal and concatenating the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature includes: Taking the modulus square of the complex function of the discrete speech signal in the frequency domain to obtain a power energy spectrum of the discrete speech signal; Extracting frequency band features of the power energy spectrum through a linear scale filter and performing discrete cosine transform to obtain linear frequency cepstrum coefficients; The peak frequency and the linear frequency cepstrum coefficient are spliced ​​according to the number of frames to obtain the fusion feature.

5. The forged voice detection method according to claim 1, characterized in that: The forged speech detection model includes a two-dimensional convolutional layer, a Res2Net module, a global average pooling layer, a fully connected layer and a softmax classification module; The step of inputting the fusion features into a trained forged speech detection model for forged speech recognition includes: Inputting the fused features into the two-dimensional convolutional layer to perform a convolution operation to extract local features; Input the local features into the Res2Net module to obtain a residual matrix; Input the residual matrix into the global average pooling layer to generate a pooled eigenvalue corresponding to each channel, and realize a linear combination of the previous layer through a fully connected layer to obtain a one-dimensional eigenvector; The one-dimensional feature vector is input into a softmax classification module to obtain a forged speech recognition result.

6. The forged voice detection method according to claim 3, characterized in that: The fitting of the frequency window and amplitude of each frame on the amplitude spectrum determines a parabola, and the frequency window corresponding to the amplitude peak is determined by performing secondary interpolation by assuming the positions of three known amplitude values, and the secondary interpolation is repeated multiple times to obtain multiple peak frequencies, including: Fitting the frequency window and amplitude of each frame on the amplitude spectrum determines a parabola: in, is the independent variable of the parabola, i.e. the frequency window, is the dependent variable of the parabola, i.e., the amplitude value, , are the parameters of the parabola; Assume that the positions of the three known amplitude values ​​are interpolated quadratically to determine the frequency window corresponding to the amplitude peak. Assume , , are three known amplitude values, and: By substituting the positions of the three amplitude values ​​into the fitted parabola, we can obtain: Calculate the frequency window corresponding to the amplitude peak: For the magnitude spectrum Frame, repeat the secondary interpolation multiple times to get multiple peak frequencies Among them, the peak frequency The calculation formula is: in, is the peak frequency, is the frequency window corresponding to the peak amplitude of the last quadratic interpolation, is the frequency window corresponding to the amplitude peak, is the speech sampling rate, is the window length.

7. The forged voice detection method according to claim 1, characterized in that: The training process of the forged speech detection model includes: Acquire a training set and a test set, wherein the training set includes a speech sample signal, and the test set includes a speech test signal; Extracting the peak frequencies of the speech sample signal and the speech test signal, extracting the linear frequency cepstral coefficients of the speech sample signal and the speech test signal, and concatenating the peak frequencies and the linear frequency cepstral coefficients to obtain a training fusion vector and a test fusion vector; Inputting the training fusion vector into the forged speech detection model to obtain a prediction result; According to the prediction result and the true value label of the voice sample signal, a binary cross entropy loss function is calculated, a gradient is calculated by back propagation, and the weights and biases of the parameters in the forged voice detection model are adjusted multiple times by iteration; The test fusion vector is input into the forged speech detection model to determine whether the prediction accuracy of the forged speech detection model reaches a preset value. If so, the training is terminated.

8. A forged voice detection device, characterized in that: include: The acquisition and preprocessing module is used to acquire the voice signal to be detected, convert the voice signal into a time-series digital signal, and preprocess the time-series digital signal to obtain a discrete voice signal; A first processing module is used to perform Fourier transform on the discrete speech signal to obtain an amplitude spectrum, perform secondary interpolation on the amplitude spectrum to obtain a peak value of each frequency window, and combine the peak values ​​to obtain a peak-frequency matrix of the discrete speech signal; A second processing module is used to extract the linear frequency cepstral coefficients of the discrete speech signal, and concatenate the peak frequency and the linear frequency cepstral coefficients to obtain a fusion feature; The recognition module is used to input the fusion features into a trained forged speech detection model to perform forged speech recognition.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the forged voice detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the forged voice detection method according to any one of claims 1 to 7.

Images