Pseudo-speech detection method, device, computer equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence, and provides a pseudo-speech detection method, device, computer equipment and storage medium. The method first obtains the speech to be detected; obtains a pre-trained neural network model, and the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers superimposed in sequence; inputs the speech to be detected into the first feature extraction network to obtain a first speech feature; inputs the first speech feature into the first residual network so that each residual layer of the first residual network is processed to obtain an output speech feature; inputs the speech feature output by the nth residual layer of the first residual network into the first classification network to obtain a classification result; and determines the authenticity of the speech to be detected based on the classification result. The embodiment of the present application utilizes a pre-trained neural network model to directly extract speech features from a speech sample, without the need to manually design acoustic features, saving time and effort.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a pseudo-speech detection method, apparatus, computer equipment and storage medium. Background Art

[0002] At present, speech synthesis and speech conversion systems based on deep learning and massive training data can synthesize speech with high naturalness. These synthesized or converted speech can almost reach the level of natural human speech. On the one hand, these synthesized or converted speech can be used for entertainment broadcasting, etc., which can save the time occupied by manual broadcasting; on the other hand, high-quality synthesized speech also increases the difficulty of speech detection. In order to detect speech more accurately, traditional speech detection usually uses acoustic features designed by experts, such as Mel cepstral coefficients, linear prediction coefficients, etc. The above methods require manual participation in calculations, which is time-consuming and laborious. Summary of the invention

[0003] The present application aims to solve the problems of the prior art to at least a certain extent, and to provide a pseudo-speech detection method, apparatus, computer equipment and storage medium, which utilizes a pre-trained neural network model to directly extract speech features from speech samples without the need for manually designed acoustic features, thus saving time and effort.

[0004] The technical solution of the embodiment of the present application is as follows:

[0005] In a first aspect, the present application provides a pseudo-speech detection method, the method comprising:

[0006] Get the voice to be detected;

[0007] Obtaining a pre-trained neural network model, wherein the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence;

[0008] Inputting the to-be-detected speech into the first feature extraction network to obtain a first speech feature;

[0009] Inputting the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, performing a first feature extraction process on the input speech feature to obtain a second speech feature, performing a first feature activation process on the input speech feature to obtain a first activation feature, performing a second feature activation process on the input speech feature to obtain a second activation feature, performing a first feature fusion process on the first activation feature and the second activation feature to obtain a fusion feature, performing a second feature extraction process on the fusion feature to obtain a third speech feature, and performing a second feature fusion process on the second speech feature and the third speech feature to obtain an output speech feature;

[0010] Inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result;

[0011] The authenticity of the speech to be detected is determined according to the classification result.

[0012] According to some embodiments of the present application, the step of inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result includes:

[0013] Performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature;

[0014] Perform linear classification processing on the fourth speech feature to obtain a classification result.

[0015] According to some embodiments of the present application, performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature includes:

[0016] Performing a first convolution process on the speech feature output by the nth residual layer to obtain a fifth speech feature;

[0017] Performing pooling processing on the fifth speech feature to obtain a pooled speech feature;

[0018] A second convolution process is performed on the pooled speech features to obtain a fourth speech feature.

[0019] According to some embodiments of the present application, performing linear classification processing on the fourth speech feature to obtain a classification result includes:

[0020] Performing linear processing on the fourth speech feature to obtain a linear feature;

[0021] Classification processing is performed on the linear features to obtain the classification result.

[0022] According to some embodiments of the present application, the pre-trained neural network model is obtained by the following steps:

[0023] Acquire a speech data set, wherein the speech data set includes a plurality of speech data;

[0024] Acquire an initial neural network model, wherein the initial neural network model includes a second feature extraction network, a second residual network, and a second classification network;

[0025] Input each of the speech data into the second feature extraction network to obtain a first speech data feature set;

[0026] Inputting the first speech data feature set into the second residual network so that the second residual network outputs a second speech data feature set;

[0027] Calculate a value of a loss function according to the second speech data feature set and the second classification network;

[0028] The initial neural network model is trained according to the value of the loss function to obtain the pre-trained neural network model.

[0029] According to some embodiments of the present application, the second classification network includes a classification feature extraction layer and a linear classification layer;

[0030] The step of calculating a value of a loss function according to the second speech data feature set and the second classification network includes:

[0031] Inputting the second speech data feature set into the classification feature extraction layer to obtain a third speech data feature set;

[0032] Calculating the angle between each speech data feature of the third speech data feature set and the weight of the linear classification layer to obtain a value of the angle;

[0033] The value of the loss function is calculated according to the value of the angle.

[0034] According to some embodiments of the present application, calculating the value of the loss function according to the value of the angle includes:

[0035] According to the value of the angle, the value of the loss function is calculated by an additional angle margin algorithm, and the calculation formula of the additional angle margin algorithm is as follows:

[0036]

[0037] Among them, L AAM represents the value of the loss function, s, N and m are all hyperparameters, θ yi,i and θ j,iRepresents the value of the angle.

[0038] In a second aspect, the present application provides a pseudo-speech detection device, comprising:

[0039] A data acquisition module, used to acquire the speech to be detected;

[0040] A model acquisition module, used to acquire a pre-trained neural network model, wherein the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence;

[0041] An input processing module, used for inputting the speech to be detected into the first feature extraction network to obtain a first speech feature;

[0042] A residual processing module, used for inputting the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, performing a first feature extraction process on the input speech feature to obtain a second speech feature, performing a first feature activation process on the input speech feature to obtain a first activation feature, performing a second feature activation process on the input speech feature to obtain a second activation feature, performing a first feature fusion process on the first activation feature and the second activation feature to obtain a fusion feature, performing a second feature extraction process on the fusion feature to obtain a third speech feature, and performing a second feature fusion process on the second speech feature and the third speech feature to obtain an output speech feature;

[0043] A classification processing module, used for inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result;

[0044] The detection processing module is used to determine the authenticity of the speech to be detected according to the classification result.

[0045] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more of the processors, the one or more processors execute the steps of any one of the methods described in the first aspect above.

[0046] In a fourth aspect, the present application also provides a computer-readable storage medium, which can be read and written by a processor, and the storage medium stores computer instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of any one of the methods described in the first aspect above.

[0047] The technical solution provided in the embodiments of the present application has the following beneficial effects:

[0048] The embodiment of the present application proposes a pseudo-speech detection method, apparatus, computer equipment and storage medium. The pseudo-speech detection method first obtains a speech to be detected for subsequent speech detection of the speech to be detected; obtains a pre-trained neural network model, the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence, which is conducive to the subsequent use of the obtained neural network model to directly extract the speech features of the speech to be detected; inputs the speech to be detected into the first feature extraction network to obtain a first speech feature, and performs feature extraction on the speech to be detected without manually designing acoustic speech features; inputs the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtains the input speech feature, performs a first classification on the input speech feature The feature extraction process is performed to obtain the second speech feature, the input speech feature is subjected to the first feature activation process to obtain the first activation feature, the input speech feature is subjected to the second feature activation process to obtain the second activation feature, the first activation feature and the second activation feature are subjected to the first feature fusion process to obtain the fusion feature, the first feature fusion process can alleviate the problem of gradient disappearance caused by network deepening, the fusion feature is subjected to the second feature extraction process to obtain the third speech feature, the second speech feature and the third speech feature are subjected to the second feature fusion process to obtain the output speech feature, the residual network can fully extract the speech features of the speech to be detected, which is beneficial to improving the detection accuracy; the speech features output by the nth residual layer of the first residual network are input into the first classification network to obtain the classification result; and then the authenticity of the speech to be detected is determined according to the classification result. The embodiment of the present application can directly extract speech features from speech using a neural network, without the need to manually design acoustic features, saving time and effort. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flowchart of a pseudo-speech detection method provided by an embodiment of the present application;

[0050] Figure 2 yes Figure 1 A schematic flow chart of a sub-step of step S500;

[0051] Figure 3 yes Figure 2 A schematic flow chart of a sub-step of step S510;

[0052] Figure 4 yes Figure 2 A schematic flow chart of a sub-step of step S520;

[0053] Figure 5is a flowchart of a pseudo-speech detection method provided by another embodiment of the present application;

[0054] Figure 6 yes Figure 5 A schematic flow chart of a sub-step of step S750;

[0055] Figure 7 It is a schematic diagram of the overall flow of a network model of a pseudo-speech detection method provided by an embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of the residual layer flow of a pseudo speech detection method provided by an embodiment of the present application;

[0057] Fig. 9 is a structural schematic diagram of a pseudo-speech detection device provided by an embodiment of the present application;

[0058] Fig.10 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] First, some nouns involved in this application are analyzed:

[0062] Neural network: In the embodiments of the present application, artificial neural networks (ANNs) are referred to as neural networks (NNs) or connection models, which are mathematical models that imitate the behavioral characteristics of animal neural networks and perform distributed parallel information processing. This network relies on the complexity of the system to adjust the interconnected relationships between a large number of internal nodes to achieve the purpose of processing information. Artificial neural networks are connected by a large number of neurons with adjustable connection weights, and have the characteristics of large-scale parallel processing, distributed information storage, and good self-organization and self-learning capabilities. The BP (Back Propagation) algorithm, also known as the error back propagation algorithm, is a supervised learning algorithm in artificial neural networks. The BP neural network algorithm can theoretically approximate any function, and its basic structure is composed of nonlinear change units, which has a strong nonlinear mapping ability. Moreover, parameters such as the number of intermediate layers of the network, the number of processing units in each layer, and the learning coefficient of the network can be set according to the specific situation, which is very flexible and has broad application prospects.

[0063] Deep learning: It is the study of the inherent laws and representation levels of sample data. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. The deep learning model is a deeper neural network. The deeper the network layers, the more difficult it is to train the deep learning model. Not only does it have higher requirements on computer hardware, but it also requires some training techniques to obtain a trained deep learning model.

[0064] Residual network: If the later layers of a deep network are identity mappings, then the model degenerates into a shallow network. How to learn the identity mapping function? If some layers are directly asked to fit a potential identity mapping function H(x)=x, this is difficult, which may be the reason why deep networks are difficult to train. However, if the network is designed as H(x)=F(x)+x, as long as F(x)=0, an identity mapping H(x)=x is formed, and fitting the residual is easier. The characteristics of residual networks are that they are easy to optimize and can improve accuracy by increasing the depth. The residual blocks inside the residual network use skip connections to alleviate the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0065] In the related art, the activation function of the neural network model usually adopts the tanh activation function and the sigmoid activation function. The sigmoid activation function is prone to oversaturation. The sigmoid activation function only has obvious gradient changes near the origin of the coordinates, and its two ends change very slowly, which will cause the gradient diffusion phenomenon when updating the parameters during the back propagation process, and this phenomenon becomes more serious as the number of network layers increases. In addition, the function output is not centrally symmetrical, and the output value of the sigmoid activation function is always greater than 0, which will cause the convergence speed of the model training to slow down. The tanh activation function is a deformation of the sigmoid activation function, which can solve the problem of slow convergence during sigmoid function training, but there is still the problem of gradient disappearance when the network deepens.

[0066] Based on this, the embodiments of the present application propose a pseudo-speech detection method, apparatus, computer device and storage medium. The pseudo-speech detection method first obtains a speech to be detected for subsequent speech detection of the speech to be detected; obtains a pre-trained neural network model, the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers superimposed in sequence, which is conducive to the subsequent use of the acquired neural network model to directly extract the speech features of the speech to be detected; inputs the speech to be detected into the first feature extraction network to obtain a first speech feature, and performs feature extraction on the speech to be detected without manually designing acoustic speech features; inputs the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtains the input speech feature, performs The first feature extraction process is performed to obtain the second speech feature, the input speech feature is subjected to the first feature activation process to obtain the first activation feature, the input speech feature is subjected to the second feature activation process to obtain the second activation feature, the first activation feature and the second activation feature are subjected to the first feature fusion process to obtain the fusion feature, the first feature fusion process can alleviate the problem of gradient disappearance caused by network deepening, the fusion feature is subjected to the second feature extraction process to obtain the third speech feature, the second speech feature and the third speech feature are subjected to the second feature fusion process to obtain the output speech feature, the residual network can fully extract the speech features of the speech to be detected, which is beneficial to improving the detection accuracy; the speech features output by the nth residual layer of the first residual network are input into the first classification network to obtain the classification result; and then the authenticity of the speech to be detected is determined according to the classification result. The embodiment of the present application can directly extract speech features from speech using a neural network, without the need to manually design acoustic features, saving time and effort.

[0067] It should be noted that the pseudo-speech detection method can be applied to detect synthesized speech or converted speech. The pseudo-speech detection method can be tested across data sets and has good generalization ability.

[0068] Embodiments of the present application can be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0069] The pseudo-speech detection method, apparatus, computer device, and storage medium provided in the embodiments of the present application are described below with reference to the accompanying drawings.

[0070] See also Figure 1 , Figure 1 The flowchart of a pseudo-speech detection method provided by an embodiment of the present application is shown. The method includes but is not limited to step S100, step S200, step S300, step S400, step S500 and step S600.

[0071] Step S100, obtaining the speech to be detected.

[0072] In one embodiment, the acquired speech to be detected may be a synthesized speech or a speech converted from a real speech. By acquiring the speech to be detected, it is convenient to perform detection processing on the speech to be detected later.

[0073] Step S200, obtaining a pre-trained neural network model, the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence.

[0074] like Figure 7As shown, the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network. Among them, the first feature extraction network is a network structure for extracting features of the speech to be detected; the first residual network is a network structure for extracting features of the extracted features output by the first feature extraction network, and the first residual network includes n residual layers stacked in sequence, and n can be 4 or 5; the first classification network is a network structure for extracting and classifying features of the extracted features output by the first residual network. By obtaining the neural network model, it is beneficial to use the neural network model for speech detection in the future.

[0075] like Figure 5 and Figure 7 As shown, the pre-trained neural network model is obtained through the following steps:

[0076] Step S710: Acquire a speech data set, where the speech data set includes a plurality of speech data.

[0077] In one embodiment, before network training, a speech data set for training and testing is obtained, and the speech data set includes multiple speech data, each of which can be synthesized speech data or converted speech data. Therefore, according to these different types of speech data, a pure synthesized speech data set, a pure converted speech data set, and a data set of a mixture of synthesized speech and converted speech can be formed. By obtaining the speech data set, it is convenient to perform subsequent neural network training and testing.

[0078] Step S720, obtaining an initial neural network model, the initial neural network model includes a second feature extraction network, a second residual network and a second classification network.

[0079] like Figure 7 As shown, the initial neural network model includes a second feature extraction network, a second residual network and a second classification network. The second feature extraction network is a network structure for extracting features of the speech to be detected; the second residual network is a network structure for extracting features of the extracted features output by the second feature extraction network; and the second classification network is a network structure for extracting and classifying features of the extracted features output by the second residual network. By obtaining the initial neural network model, it is beneficial to subsequently train the initial neural network model.

[0080] Step S730, input each voice data into the second feature extraction network to obtain a first voice data feature set.

[0081] In one embodiment, the second feature extraction network is used to extract features from each voice data to obtain a first voice data feature set, which can process the input voice data sample and facilitate subsequent feature extraction processing using the first voice data feature set. The first voice data feature set is a collection of voice data features obtained by the second feature extraction network.

[0082] It should be noted that the second feature extraction network can be a convolutional neural network, a fully connected neural network, or other networks capable of extracting features from speech data, which will not be described in detail here. The second feature extraction network is used to extract features from input speech data and can convert speech data vectors into extracted feature vectors.

[0083] Step S740: input the first speech data feature set into the second residual network so that the second residual network outputs the second speech data feature set.

[0084] In one embodiment, the first speech data feature set is input into the second residual network, the second residual network includes multiple residual layers, the first residual layer processes the first speech data feature set, and the subsequent nth residual layer processes the speech data feature set output by the n-1th residual layer to obtain the second speech data feature set output by the nth residual layer. The second speech data feature set is a collection of speech data features obtained by the residual network. Using the second residual network for feature extraction can fully extract speech features and increase the accuracy of speech detection.

[0085] Step S750, calculating the value of the loss function according to the second speech data feature set and the second classification network.

[0086] like Figure 6 As shown, the second classification network includes a classification feature extraction layer and a linear classification layer. According to the second speech data feature set and the second classification network, the value of the loss function is calculated, including but not limited to the following steps:

[0087] Step S751, input the second speech data feature set into the classification feature extraction layer to obtain the third speech data feature set.

[0088] In one embodiment, the second voice data feature set is input into the classification feature extraction layer, and the classification feature extraction layer is used to extract the features of each of the two voice data features of the second voice data feature set to obtain a third voice data feature set, which is beneficial for the subsequent calculation of the loss function value using the third voice data feature set. The third voice data feature set is a collection of voice data features obtained by the classification feature extraction layer.

[0089] It should be noted that the classification feature extraction layer can be a convolution layer or a fully connected layer. Exemplarily, the parameters of the convolution layer can be set as follows: the size of the convolution kernel is a 1*1 convolution kernel, the number of channels is 256, and the dimension of the intermediate latent variable is 1*256.

[0090] Step S752, calculating the angle between each speech data feature of the third speech data feature set and the weight of the linear classification layer to obtain the value of the angle.

[0091] In one embodiment, according to each speech data feature vector in the third speech data feature set, the angle between each speech data feature vector and the weight of the linear classification layer is calculated, wherein the weight of the linear classification layer refers to the weight of the input layer in the linear classification layer, and the value of the angle is obtained. According to the value of the angle, it is convenient to perform subsequent calculation processing of the loss function.

[0092] Step S753, calculating the value of the loss function according to the value of the angle.

[0093] In one embodiment, according to the value of the included angle, the value of the loss function is calculated by an additional angle margin algorithm, and the calculation formula of the additional angle margin algorithm is as follows:

[0094]

[0095] Among them, L AAM represents the value of the loss function, s, N and m are all hyperparameters, θ yi,i and θ j,i Indicates the value of the angle.

[0096] Step S760, training the initial neural network model according to the value of the loss function to obtain a pre-trained neural network model.

[0097] In one embodiment, when the value of the loss function satisfies the preset training end condition, the training of the initial neural network model is terminated to obtain a pre-trained neural network model, which is used for subsequent detection and processing of the speech to be detected. The training end condition can be that the value of the loss function is less than the preset loss value, and the training is terminated; it can also be the number of training iterations as the training end condition. For example, the preset number of training times is 10,000 times, and the training is terminated when the loop parameter reaches the preset number of training times; it can also be other training end conditions, as long as the training can be terminated.

[0098] In one embodiment, when the value of the loss function does not meet the preset training end condition, that is, the value of the loss function cannot end the training after multiple trainings, the number of training iterations is used as the training end condition. When the preset number of trainings is reached, regardless of whether the value of the loss function meets the preset training end condition, the training is ended to obtain a pre-trained neural network model, which is used for subsequent detection and processing of the speech to be detected. By combining different training end conditions, the infinite loop problem can be avoided, wherein multiple times means that the number of trainings is greater than 100,000, and this value can be modified according to actual conditions. Other combinations can also be used as long as the infinite loop problem can be avoided.

[0099] It should be noted that the initial neural network model can be continuously trained by adjusting the hyperparameters of the initial neural network model to find a set of hyperparameters that can achieve a higher model detection accuracy. The pre-trained neural network model corresponding to this set of hyperparameters has a higher accuracy.

[0100] Step S300: input the speech to be detected into a first feature extraction network to obtain a first speech feature.

[0101] In one embodiment, the first feature extraction network is used to extract features from the speech to be detected to obtain the first speech feature, which can process the input speech sample to be detected, and is conducive to the subsequent feature extraction calculation processing using the first speech feature. The first speech feature is a feature vector obtained by the first feature extraction network.

[0102] Step S400, input the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtain the input speech feature, perform a first feature extraction process on the input speech feature to obtain a second speech feature, perform a first feature activation process on the input speech feature to obtain a first activation feature, perform a second feature activation process on the input speech feature to obtain a second activation feature, perform a first feature fusion process on the first activation feature and the second activation feature to obtain a fusion feature, perform a second feature extraction process on the fusion feature to obtain a third speech feature, and perform a second feature fusion process on the second speech feature and the third speech feature to obtain an output speech feature.

[0103] like Figure 8As shown, according to step S300, the first speech feature obtained is input into the first residual network, and the first residual network includes a plurality of residual layers, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, performing the first feature extraction processing on the input speech feature to obtain the second speech feature, performing the first feature activation processing on the input speech feature to obtain the first activation feature, and the input speech feature can be normalized to avoid the exponential growth of calculation; performing the second feature activation processing on the input speech feature to obtain the second activation feature, and the input speech feature can be normalized to avoid the exponential growth of calculation; performing the first feature fusion processing on the first activation feature and the second activation feature to obtain the fusion feature, and the problem of gradient disappearance caused by network deepening can be alleviated by obtaining the fusion feature; performing the second feature extraction processing on the fusion feature to obtain the third speech feature, and performing the second feature fusion processing on the second speech feature and the third speech feature to obtain the output speech feature. Each residual layer performs the above processing to fully extract the speech feature, thereby increasing the accuracy of detection.

[0104] It should be noted that when the input speech feature is obtained, when the first residual layer is processed, the speech feature is the first speech feature, and the subsequent nth residual layer processes the speech feature output by the n-1th residual layer to obtain the speech feature output by the nth residual layer. After multiple residual layer processing, feature extraction can be fully performed, thereby increasing the accuracy of detection.

[0105] It should be noted that the first feature extraction processing and the second feature extraction processing are both feature extraction processing methods using convolutional layers, fully connected layers or other feature extraction layers; the first feature activation processing and the second feature activation processing are both processing methods using tanh activation function, sigmoid activation function or relu activation function to perform feature normalization; the first feature fusion processing and the second feature fusion processing can both be fusion processing methods of feature vector addition or feature vector dot product; the first activation feature and the second activation feature are both feature vectors obtained through activation function processing; the second speech feature and the third speech feature are feature vectors obtained through feature extraction processing.

[0106] Step S500, inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result.

[0107] In one embodiment, the first classification network is a network structure that extracts and classifies the extracted features output by the first residual network. The speech features output by the nth residual layer of the first residual network are input into the first classification network to obtain a classification result, which is a probability value indicating the probability of the authenticity of the speech to be detected, which is conducive to the subsequent determination of the authenticity of the speech to be detected according to the classification result.

[0108] like Figure 2 As shown, the speech features output by the nth residual layer of the first residual network are input into the first classification network to obtain the classification result, including but not limited to the following steps:

[0109] Step S510, performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature.

[0110] like Figure 3 and Figure 7 As shown, performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature includes but is not limited to the following steps:

[0111] Step S511, performing a first convolution process on the speech feature output by the nth residual layer to obtain a fifth speech feature.

[0112] In one embodiment, the speech feature output by the nth residual layer is input to the convolution layer for the first convolution process to obtain the fifth speech feature. By performing the first convolution process, the features output by the first residual network are integrated to achieve full feature extraction. The first convolution process is a processing operation performed by the convolution layer; the fifth speech feature is a feature vector obtained by processing the features output by the residual layer by the convolution layer.

[0113] Step S512: performing pooling processing on the fifth speech feature to obtain a pooled speech feature.

[0114] In one embodiment, according to step S511, the fifth speech feature is pooled, and the pooling may be performed using the maximum value, the mean value, or the statistical pooling to obtain the pooled speech feature. By performing the pooling process on the extracted features, the feature dimension can be reduced and the receptive field can be increased.

[0115] Step S513: Perform a second convolution process on the pooled speech features to obtain fourth speech features.

[0116] In one embodiment, according to step S512, the obtained pooled speech features are subjected to a second convolution process to obtain a fourth speech feature, and feature extraction is performed on the feature vector that increases the receptive field to achieve full feature extraction. The second convolution process is a processing operation performed using a convolution layer; the fourth speech feature is a feature vector obtained by processing the pooled features using a convolution layer.

[0117] Step S520: Perform linear classification processing on the fourth speech feature to obtain a classification result.

[0118] like Figure 4 and Figure 7 As shown, linear classification is performed on the fourth speech feature to obtain a classification result, including but not limited to the following steps:

[0119] Step S521, performing linear processing on the fourth speech feature to obtain a linear feature.

[0120] In one embodiment, the fourth speech feature obtained in step S510 is linearly processed using a linear layer to obtain a linear feature. The linear layer may be a fully connected layer or a 1*1 convolutional layer. The linear processing can obtain edge features of the speech to be detected and achieve full feature extraction.

[0121] Step S522: classify the linear features to obtain classification results.

[0122] In one embodiment, the linear features obtained in step S521 are classified and processed. The softmax classification layer can be used to classify the linear features, or the sigmoid classification layer can be used to classify the linear features to obtain a classification result. The classification result is a probability value. When the classification result is greater than the preset probability value, the speech to be detected corresponding to the classification result greater than the preset probability value is classified into one category. When the classification result is less than or equal to the preset probability value, the speech to be detected corresponding to the classification result less than or equal to the preset probability value is classified into one category. This is conducive to the subsequent determination of the authenticity of the speech to be detected based on the classification result. Among them, the preset probability value can be 0.6 or 0.7, which can be modified according to needs and will not be described here.

[0123] Step S600: determining the authenticity of the speech to be detected according to the classification result.

[0124] In one embodiment, according to step S500, the classification result is a probability value. When the classification result corresponding to the speech to be detected is greater than a preset probability value, the speech to be detected is determined to be a real speech; when the classification result corresponding to the speech to be detected is less than or equal to the preset probability value, the speech to be detected is determined to be a forged speech, thereby distinguishing the authenticity of the speech to be detected.

[0125] like Fig. 9As shown, an embodiment of the present application provides a pseudo-speech detection device 100, which includes using a data acquisition module 110 to acquire a speech to be detected, for subsequent speech detection of the speech to be detected; using a model acquisition module 120 to acquire a pre-trained neural network model, the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence, which is conducive to the subsequent use of the acquired neural network model to directly extract the speech features of the speech to be detected; inputting the speech to be detected into the first feature extraction network through an input processing module 130 to obtain a first speech feature, and extracting features from the speech to be detected without manually designing acoustic speech features; and then using a residual processing module 140 to input the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, The first feature extraction process is performed on the input voice feature to obtain the second voice feature, the first feature activation process is performed on the input voice feature to obtain the first activation feature, the second feature activation process is performed on the input voice feature to obtain the second activation feature, the first activation feature and the second activation feature are subjected to the first feature fusion process to obtain the fusion feature, the first feature fusion process can alleviate the problem of gradient disappearance caused by network deepening, the fusion feature is subjected to the second feature extraction process to obtain the third voice feature, the second voice feature and the third voice feature are subjected to the second feature fusion process to obtain the output voice feature, and the residual network can fully extract the voice features of the voice to be detected, which is conducive to improving the detection accuracy; then the classification processing module 150 is used to input the voice features output by the nth residual layer of the first residual network into the first classification network to obtain the classification result; finally, the detection processing module 160 is used to determine the authenticity of the voice to be detected according to the classification result. The embodiment of the present application can directly extract voice features from the voice using a neural network, without the need to manually design acoustic features, saving time and effort.

[0126] It should be noted that the data acquisition module 110 is connected to the model acquisition module 120, the model acquisition module 120 is connected to the input processing module 130, the input processing module 130 is connected to the residual processing module 140, the residual processing module 140 is connected to the classification processing module 150, and the classification processing module 150 is connected to the detection processing module 160. The above-mentioned pseudo-speech detection method acts on the pseudo-speech detection device 100, and the pseudo-speech detection device 100 can directly extract speech features from speech samples using a pre-trained neural network model, without the need to manually design acoustic features, saving time and effort.

[0127] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0128] Fig.10 The computer device 500 provided in the embodiment of the present application is shown. The computer device 500 may be a server or a terminal, and the internal structure of the computer device 500 includes but is not limited to:

[0129] A memory 510, used for storing programs;

[0130] The processor 520 is used to execute the program stored in the memory 510. When the processor 520 executes the program stored in the memory 510, the processor 520 is used to execute the above-mentioned pseudo-speech detection method.

[0131] The processor 520 and the memory 510 may be connected via a bus or other means.

[0132] The memory 510 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs and non-transitory computer executable programs, such as the pseudo-speech detection described in any embodiment of the present application. The processor 520 implements the above-mentioned pseudo-speech detection by running the non-transitory software programs and instructions stored in the memory 510.

[0133] The memory 510 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store and execute the above-mentioned pseudo-speech detection. In addition, the memory 510 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 510 may optionally include a memory remotely arranged relative to the processor 520, and these remote memories may be connected to the processor 520 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0134] The non-transient software programs and instructions required to implement the above-mentioned pseudo-speech detection are stored in the memory 510, and when executed by one or more processors 520, the pseudo-speech detection provided by any embodiment of the present application is performed.

[0135] The embodiment of the present application also provides a computer-readable storage medium storing computer-executable instructions, where the computer-executable instructions are used to perform the above-mentioned pseudo-speech detection.

[0136] In one embodiment, the storage medium stores computer executable instructions, which are executed by one or more control processors 520, for example, by a processor 520 in the above-mentioned computer device 500, so that the above-mentioned one or more processors 520 can perform pseudo-speech detection provided by any embodiment of the present application.

[0137] The above described embodiments are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0138] The terms "first", "second", "third", "fourth", "fifth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0139] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0140] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0141] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A pseudo-speech detection method, characterized in that: The method comprises: Get the voice to be detected; Obtaining a pre-trained neural network model, wherein the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence; Inputting the to-be-detected speech into the first feature extraction network to obtain a first speech feature; Inputting the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, performing a first feature extraction process on the input speech feature to obtain a second speech feature, performing a first feature activation process on the input speech feature to obtain a first activation feature, performing a second feature activation process on the input speech feature to obtain a second activation feature, performing a first feature fusion process on the first activation feature and the second activation feature to obtain a fusion feature, performing a second feature extraction process on the fusion feature to obtain a third speech feature, and performing a second feature fusion process on the second speech feature and the third speech feature to obtain an output speech feature; Inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result; The authenticity of the speech to be detected is determined according to the classification result.

2. The method according to claim 1, characterized in that: The step of inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result comprises: Performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature; Perform linear classification processing on the fourth speech feature to obtain a classification result.

3. The method according to claim 2, characterized in that The performing a third feature extraction process on the speech feature output by the nth residual layer to obtain a fourth speech feature comprises: Performing a first convolution process on the speech feature output by the nth residual layer to obtain a fifth speech feature; Performing pooling processing on the fifth speech feature to obtain a pooled speech feature; A second convolution process is performed on the pooled speech features to obtain a fourth speech feature.

4. The method according to claim 2, characterized in that: The performing linear classification processing on the fourth speech feature to obtain a classification result includes: Performing linear processing on the fourth speech feature to obtain a linear feature; Classification processing is performed on the linear features to obtain the classification result.

5. The method according to claim 1, characterized in that The pre-trained neural network model is obtained by the following steps: Acquire a speech data set, wherein the speech data set includes a plurality of speech data; Acquire an initial neural network model, wherein the initial neural network model includes a second feature extraction network, a second residual network, and a second classification network; Input each of the speech data into the second feature extraction network to obtain a first speech data feature set; Inputting the first speech data feature set into the second residual network so that the second residual network outputs a second speech data feature set; Calculate a value of a loss function according to the second speech data feature set and the second classification network; The initial neural network model is trained according to the value of the loss function to obtain the pre-trained neural network model.

6. The method according to claim 5, characterized in that The second classification network includes a classification feature extraction layer and a linear classification layer; The step of calculating a value of a loss function according to the second speech data feature set and the second classification network includes: Inputting the second speech data feature set into the classification feature extraction layer to obtain a third speech data feature set; Calculating the angle between each speech data feature of the third speech data feature set and the weight of the linear classification layer to obtain a value of the angle; The value of the loss function is calculated according to the value of the angle.

7. The method according to claim 6, characterized in that The calculating the value of the loss function according to the value of the angle includes: According to the value of the angle, the value of the loss function is calculated by an additional angle margin algorithm, and the calculation formula of the additional angle margin algorithm is as follows: Among them, L AAM represents the value of the loss function, s, N and m are all hyperparameters, θ yi,i and θ j,i Represents the value of the angle.

8. A pseudo-speech detection device, characterized in that: include: A data acquisition module, used to acquire the speech to be detected; A model acquisition module, used to acquire a pre-trained neural network model, wherein the pre-trained neural network model includes a first feature extraction network, a first residual network and a first classification network, wherein the first residual network includes n residual layers stacked in sequence; An input processing module, used for inputting the speech to be detected into the first feature extraction network to obtain a first speech feature; A residual processing module, used for inputting the first speech feature into the first residual network, so that each residual layer of the first residual network performs the following processing respectively: obtaining the input speech feature, performing a first feature extraction process on the input speech feature to obtain a second speech feature, performing a first feature activation process on the input speech feature to obtain a first activation feature, performing a second feature activation process on the input speech feature to obtain a second activation feature, performing a first feature fusion process on the first activation feature and the second activation feature to obtain a fusion feature, performing a second feature extraction process on the fusion feature to obtain a third speech feature, and performing a second feature fusion process on the second speech feature and the third speech feature to obtain an output speech feature; A classification processing module, used for inputting the speech features output by the nth residual layer of the first residual network into the first classification network to obtain a classification result; The detection processing module is used to determine the authenticity of the speech to be detected according to the classification result.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more of the processors, the one or more processors execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium can be read and written by a processor, and the storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method as described in any one of claims 1 to 7.

Images