Method, device, electronic device and storage medium for detecting spoofed speech

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for detecting fake speech. The method includes: determining the speech to be detected; inputting the speech into a speech detection model to obtain a speech detection result output by the speech detection model, wherein the speech detection result is either real speech or fake speech; wherein the speech detection model is obtained by comparative training based on sample speech pairs, the sample speech pairs including real speech samples and fake speech samples, and the real speech samples and fake speech samples have the same text information and acoustic features. This improves the accuracy and versatility of speech authenticity detection.

Description

Technical Field

[0001] This invention relates to the field of speech detection technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting fake speech. Background Technology

[0002] Automatic speaker verification (ASV), a biometric technology, can determine whether the currently input voice belongs to a specific speaker registered with the system, thereby achieving identity authentication. Due to its convenience and seamless operation, it is increasingly widely used in access control, e-commerce, and smart home scenarios. However, the development of voice spoofing technologies, such as text-to-speech (TTS) and voice conversion (VC), has threatened the security of speech recognition.

[0003] Currently, methods for detecting fake speech mainly use the original speech waveform or extract various acoustic features through time-frequency domain changes as front-end input. Then, a model capable of distinguishing between genuine and fake speech is built based on supervised binary classification training to achieve speech authenticity detection. However, because it learns the distinctiveness of pronunciation in speech, its adaptability is low, resulting in poor accuracy and performance in scenarios with low adaptability. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for detecting fake voice, in order to solve the problem of poor adaptability and accuracy of fake voice detection and recognition in the prior art.

[0005] This invention provides a method for detecting fake voice, comprising:

[0006] Identify the speech to be detected;

[0007] The speech is input into the speech detection model to obtain the speech detection result output by the speech detection model. The speech detection result is either real speech or fake speech.

[0008] The speech detection model is obtained by comparative training based on sample speech pairs, which include real speech samples and fake speech samples. The real speech samples and the fake speech samples have the same text information and acoustic features.

[0009] According to a method for detecting fake speech provided by the present invention, the training steps of the speech detection model include:

[0010] Determine the initial pre-trained model and sample speech pairs;

[0011] Based on the initial pre-trained model, feature extraction is performed on the sample speech pairs to obtain the first feature of the real speech sample and the second feature of the fake speech sample;

[0012] A loss function is constructed based on the first feature and the second feature, and the parameters of the initial pre-trained model are iterated based on the loss function to obtain a speech pre-trained model;

[0013] An initial detection model is constructed based on the model parameters of the pre-trained speech model, and the initial detection model is optimized based on the sample speech pairs to obtain a speech detection model.

[0014] According to a method for detecting fake speech provided by the present invention, the step of extracting features from the sample speech pairs to obtain a first feature of the real speech sample and a second feature of the fake speech sample includes:

[0015] Feature extraction is performed on the real speech sample to obtain the first convolutional feature of the real speech sample, and feature extraction is performed on the fake speech sample to obtain the second convolutional feature of the fake speech sample;

[0016] The first convolutional feature and the second convolutional feature are masked respectively, and the first convolutional feature and the second convolutional feature after masking are extracted to obtain the first context feature and the second context feature.

[0017] The first convolutional feature and the second convolutional feature are quantized to obtain the first quantized feature and the second quantized feature;

[0018] The first feature is determined based on the first context feature and the first quantization feature, and the second feature is determined based on the second context feature and the second quantization feature.

[0019] According to a method for detecting fake speech provided by the present invention, the step of masking the first convolutional feature and the second convolutional feature includes:

[0020] Determine the text information corresponding to the sample speech pair, and perform word segmentation on the text information;

[0021] Select several words from the words obtained from word segmentation to form a first word set, and determine the time step of each word in the first word set;

[0022] Masking is performed on the first convolutional feature and the second convolutional feature according to the time step.

[0023] According to a method for detecting fake speech provided by the present invention, the step of constructing a loss function based on the first feature and the second feature includes:

[0024] The contrast loss function is determined based on the first context feature, the second context feature, the first quantization feature, and the second quantization feature;

[0025] Determine the context feature loss function based on the first context feature and the second context feature;

[0026] The loss function of the initial pre-trained model is determined based on the contrast loss function and the context feature loss function.

[0027] According to a method for detecting fake speech provided by the present invention, the step of optimizing the initial detection model based on the sample speech to obtain a speech detection model includes:

[0028] Based on the initial detection model, speech detection is performed on the real speech samples and the fake speech samples respectively, and an optimized loss function is determined based on the obtained sample speech detection results;

[0029] The initial detection model is iterated based on the optimized loss function to obtain the speech detection model.

[0030] According to a method for detecting fake speech provided by the present invention, the fake speech sample is obtained by fusing the text information of the real speech sample with the acoustic features of the real speech sample, wherein the acoustic features include at least timbre features and voiceprint features.

[0031] The present invention also provides a device for detecting fake voice, comprising:

[0032] The speech acquisition module is used to determine the speech to be detected;

[0033] The speech detection module is used to input the speech into the speech detection model and obtain the speech detection result output by the speech detection model. The speech detection result is one of real speech and fake speech.

[0034] The speech detection model is obtained by comparative training based on sample speech pairs, which include real speech samples and fake speech samples. The real speech samples and the fake speech samples have the same text information and acoustic features.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the forged voice detection methods described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the forged speech detection methods described above.

[0037] The forged speech detection method, apparatus, electronic device, and storage medium provided by this invention, when constructing forged speech in the sample speech pairs for training, ensure that the text information and acoustic features of the real speech samples and forged speech samples in each sample speech pair are the same. This allows the obtained speech detection model to perform authenticity detection and judgment not by learning pronunciation prediction tasks, but by using the distinction between real speech and forged speech as the optimization objective, extracting the intrinsic information for distinguishing the authenticity of speech, and improving the accuracy and versatility of speech authenticity detection. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the fake voice detection method provided by the present invention;

[0040] Figure 2 This is a flowchart illustrating the steps for obtaining a speech detection model provided by the present invention;

[0041] Figure 3 This is a flowchart illustrating the steps for obtaining the first feature and the second feature provided by the present invention;

[0042] Figure 4 This is a schematic diagram of the structure of the initial pre-trained model provided by the present invention;

[0043] Figure 5 This is a flowchart illustrating the steps for constructing the loss function provided by the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of the initial detection model provided by the present invention;

[0045] Figure 7 This is a schematic diagram of the structure of the fake voice detection device provided by the present invention;

[0046] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] Existing speech forgery detection technologies typically use real natural human speech and forged speech synthesized by forgery algorithms as training sets, and obtain a model that can distinguish between real and fake speech based on supervised binary classification training.

[0049] However, the fake speech detection model obtained through this binary classification training still suffers from inaccurate detection, limiting its effectiveness. This is mainly because binary classification training requires pre-extracting speech features and training directly with the classification target as the guide. However, acoustic features contain rich attribute information, including voiceprint, gender, and channel information. This means that with limited training data, this direct binary classification training often fails to fully extract the intrinsic information of fake speech.

[0050] The main problem with the above methods is that the speech detection model has poor stability and versatility. After slightly modifying the forged speech, such as adding noise or changing the speed, the detection effect will drop sharply.

[0051] To address the above problems, embodiments of the present invention provide a method for detecting fake voice. Figure 1 This is a flowchart illustrating the forged voice detection method provided by the present invention. Figure 1 The method includes:

[0052] Step 101: Determine the speech to be detected.

[0053] When performing voice verification, the voice to be detected is first identified, and then its authenticity is checked according to the selected voice detection method. Specifically, the voice to be detected can be user-inputted voice or voice output by machine devices during interaction.

[0054] Step 102: Input the speech into the speech detection model to obtain the speech detection result output by the speech detection model. The speech detection result is either real speech or fake speech.

[0055] The speech detection model is trained by comparing sample speech pairs, which include real speech samples and fake speech samples. The text information and acoustic features of real speech samples and fake speech samples are the same.

[0056] Specifically, after identifying the speech to be detected, the speech is detected using a pre-trained speech detection model to output a corresponding speech detection result. This result is either real speech or fabricated speech. Real speech is the speech produced by a human, while fabricated speech is obtained through speech synthesis or other processing techniques.

[0057] To achieve speech authenticity detection, the constructed speech detection model needs to be trained in advance. During training, corresponding sample speech pairs are pre-constructed, and the model is trained based on these sample speech pairs. Each sample speech pair contains both real and fake speech, and both real and fake speech have the same textual information and acoustic features. This highlights the differences in intrinsic information between real and fake speech during training, thereby improving the accuracy of speech detection.

[0058] Reference Figure 2 , Figure 2 This is a flowchart illustrating the steps for obtaining a speech detection model provided by the present invention, wherein the steps include steps 201 to 204.

[0059] Step 201: Determine the initial pre-trained model and sample speech pairs;

[0060] Step 202: Based on the initial pre-trained model, extract features from the sample speech pairs to obtain the first feature of the real speech sample and the second feature of the fake speech sample.

[0061] Step 203: Construct a loss function based on the first feature and the second feature, and iterate the parameters of the initial pre-trained model based on the loss function to obtain the speech pre-trained model;

[0062] Step 204: Construct an initial detection model based on the model parameters of the pre-trained speech model, and optimize the initial detection model based on sample speech pairs to obtain the speech detection model.

[0063] Specifically, during training, an initial pre-training model and sample speech pairs are first determined. Then, the sample speech pairs are used to train the initial pre-training model. During training, features are extracted from the sample speech pairs based on the initial pre-training model to obtain the first feature of the real speech sample and the second feature of the fake speech sample. Next, a corresponding loss function is constructed based on the obtained first and second features. Then, the parameters of the initial pre-training model are iterated based on the obtained loss function to obtain the speech pre-training model. Finally, the initial detection model constructed based on the model parameters of the speech pre-training model is optimized to obtain the speech detection model.

[0064] The initial pre-trained model is a speech pre-trained model. Each sample speech pair consists of a real speech and a fake speech, with the text information and acoustic features of both being identical. When generating the fake speech from the real speech, the text information and acoustic features of the real speech are fused together. The acoustic features include at least timbre and voiceprint features. A sample speech pair can be represented as (x... i ,y i ), where x i For real voice, y i To forge voice.

[0065] For example, for the initial pre-trained model, the feature information of the speech can be extracted, and then the initial pre-trained model can be trained based on the obtained features. Some model parameters in the speech pre-trained model obtained after training will be used to build the initial detection model. This is achieved by setting the model parameters of the same structural parts as those in the speech pre-trained model when building the initial detection model.

[0066] Therefore, before constructing and training the speech detection model, a pre-trained speech model is obtained. By adopting the speech pre-training strategy, prior knowledge can be learned from large-scale data in advance, providing a stable foundation for subsequent optimization and training, which can effectively improve the stability of the speech detection model.

[0067] Furthermore, when training the initial pre-trained model to obtain the speech pre-trained model, the first and second features corresponding to real speech samples and forged speech samples are obtained through feature extraction, respectively, as the basis for optimizing the initial pre-trained model. When obtaining the first and second features, features are acquired from real and forged speech samples according to the requirements of feature extraction, referring to... Figure 3 , Figure 3 This is a flowchart illustrating the steps for obtaining the first feature and the second feature provided by the present invention, wherein the steps include steps 301 to 304.

[0068] Step 301: Extract features from real speech samples to obtain the first convolutional features of real speech samples, and extract features from fake speech samples to obtain the second convolutional features of fake speech samples.

[0069] Step 302: Mask the first convolutional feature and the second convolutional feature respectively, and extract context features from the masked first convolutional feature and the second convolutional feature respectively to obtain the first context feature and the second context feature.

[0070] Step 303: Quantize the first convolutional feature and the second convolutional feature to obtain the first quantized feature and the second quantized feature;

[0071] Step 304: Determine the first feature based on the first context feature and the first quantization feature, and determine the second feature based on the second context feature and the second quantization feature.

[0072] When extracting features from real and fake speech samples in a sample speech pair, the extraction of speech features is implemented according to the requirements and the constructed model structure. Specifically, during feature extraction, coarse feature extraction is first performed on the real and fake speech samples to obtain first convolutional features and second convolutional features, respectively. Then, the obtained first and second convolutional features are masked, and then context features are extracted from the masked first and second convolutional features to obtain first and second context features, respectively. At the same time, the obtained first and second convolutional features are quantized to obtain first and second quantized features, where the first feature includes the first quantized feature and the first context feature, and the second feature includes the second quantized feature and the second context feature.

[0073] For example, the model structure corresponding to the constructed initial pre-trained model can be as follows: Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of the initial pre-trained model provided by the present invention. The model includes a convolution module, a masking module, a context module, a quantization module, and a contrast module. When a sample speech pair is input into the initial pre-trained model, the convolution module in the model extracts convolutional features from the speech. Then, the masking module, the context module, and the quantization module respectively process and extract the filtered features, highlighting the features used to determine the authenticity of the speech. Finally, the contrast module optimizes and adjusts the parameters of each part.

[0074] For each part of the initial pre-trained model, different network structures can be set according to actual needs. For example, for the convolutional module for extracting convolutional features, it can be set as a 7-layer one-dimensional convolutional structure, with each layer having a convolutional dimension of 512, a stride of [5,2,2,2,2,2,2], and a kernel of [10,3,3,3,3,2,2]. The duration of the original speech corresponding to each frame in the extracted convolutional features can also be set to 20ms. In this case, the convolutional feature corresponding to a sample speech pair after convolutional processing is (X... i ,Y i ).

[0075] For example, the context module can be configured to consist of multiple Transformer modules, each containing an FNN, LayerDrop, and Attention structure. The number of Transformer modules could be set to 12, with an inner FNN dimension of 3072, 8 Attention heads, and an output dimension of 768 for the encoding layer. The output obtained through the context module is... Contextual features of real speech, Contextual features for forged speech.

[0076] Of course, different network structures can be set for all parts according to actual needs, without any specific restrictions.

[0077] Furthermore, as described above, after obtaining the first convolutional feature and the second convolutional feature, when extracting the corresponding contextual features, it is necessary to perform masking processing on the obtained convolutional features in advance, including masking the first convolutional feature and masking the second convolutional feature. At the same time, when performing masking processing, the parts and regions of the first convolutional feature and the second convolutional feature are the same.

[0078] Specifically, the masking process includes: determining the text information corresponding to the sample speech pair and performing word segmentation on the text information; selecting several words from the words obtained from the word segmentation to form a first word set and determining the time step of each word in the first word set; and masking the first convolutional feature and the second convolutional feature according to the time step.

[0079] During masking, the parts to be masked are determined to be the same. This can be achieved by identifying the text information corresponding to the sample speech pairs, and then determining the masking parts within the identified text information, thereby masking the first and second convolutional features. Specifically, the text information of each sample speech pair is acquired, and then word segmentation is performed on the text information to obtain the word segmentation results for each sample speech pair, such as a word set. Then, a certain number of words are selected as masking objects from the segmented words, and the time step of the selected words is determined. Finally, the first and second convolutional features are masked according to the time step.

[0080] For example, the mask is used to obtain the mask position required by the context module. For the input sample speech pair, the recognition text result can be obtained by an automatic speech recognition system, or the corresponding text information can be predetermined. For the corresponding text information, the start and end times of each word in the text can be determined.

[0081] Since real and fake speech are a set of speech with identical content and voiceprint features, after segmenting the text information into words, a certain number of words can be randomly selected from the resulting words (e.g., N words) as the masking targets. For example, N / 2 (half) words can be selected as the masking targets, and then the masking operation can be performed on the obtained convolutional features based on the positions of the selected words. The position of each word can be represented using time steps to determine the position of each word. The time step is information in temporal order. For any speech, each word has a certain temporal order relationship. The time step can be determined based on the timestamp or other information, such as the sequence number label of each word in the corresponding speech text.

[0082] Furthermore, according to the above description, after the masking processing of the first convolutional feature and the second convolutional feature is completed, they are input into the context module for further extraction of feature information to obtain the corresponding first context feature and second context feature.

[0083] Simultaneously, when quantizing the first and second convolutional features, they are input into the quantization module to output the corresponding first and second quantized features. During processing, the quantization module concatenates representation vectors obtained from different codebooks and uses a linear transformation layer to obtain the final representation. The Gumbelsoftmax function is used to select entries from the codebook while maintaining full differentiability. Here, the quantization module can be configured with 2 codebooks, 320 entries, and each entry having a dimension of 128. The resulting quantized features are:

[0084] Furthermore, during the training of the initial pre-trained model, training is completed based on sample speech pairs. In the specific training process, a relevant loss function is generated based on the obtained features to iteratively optimize the model parameters, resulting in the speech pre-trained model upon completion of iterative optimization. The determination of whether iterative optimization is complete can be based on the generated loss function, which is constructed from the first and second features during the training process of the initial pre-trained model.

[0085] Reference Figure 5 , Figure 5 This is a flowchart illustrating the steps for constructing a loss function provided by the present invention, wherein the steps include steps 501 to 503.

[0086] Step 501: Determine the contrastive loss function based on the first context feature, the second context feature, the first quantization feature, and the second quantization feature;

[0087] Step 502: Determine the context feature loss function based on the first context feature and the second context feature;

[0088] Step 503: Determine the loss function of the initial pre-trained model based on the contrast loss function and the context feature loss function.

[0089] Specifically, when constructing the loss function, a contrastive loss function is determined based on the first context feature, the second context feature, the first quantization feature, and the second quantization feature. At the same time, a context feature loss function is determined based on the first context feature and the second context feature. Finally, based on the two loss functions obtained, the loss function of the initial pre-trained model during the training process is obtained.

[0090] In fact, when constructing the contrastive loss function, the contrastive loss function generated during training based on sample speech pairs is determined through contrast processing. At the same time, in order to better distinguish the contextual features of real speech and fake speech, a loss function is also constructed based on the contextual features. Then, the loss function of the model is determined based on the two loss functions obtained.

[0091] For example, the contextual representation of a sample speech pair and The combination is C, the quantization representation combination is Q, and the objects masked during masking include both real and fake speech, with a mask length of K. During training, first, time step t is taken, and the output of the context module is taken from C as c. t For comparison task loss m Taking the value at time step t from Q as a positive example, and considering other quantization representations outside time step t as negative examples, the correct quantization hidden layer speech representation needs to be distinguished from K quantization candidate representations. The contrastive prediction loss function can then be defined as:

[0092]

[0093] Where, sim(a,b)=a T b / ||a||||b|| is the cosine similarity between the contextual representation and the quantized hidden layer speech representation.

[0094] Since both positive and negative examples are included in the masking time step, and the text content and voiceprint attributes of the masking time step are the same for both real and fake speech, this loss function is not for learning pronunciation classification tasks, but for learning how to correctly distinguish between real and fake speech.

[0095] In addition, to better distinguish the contextual representations of real speech and fake speech, a minimum mean square error loss function can be used as the loss function during model training. The minimum mean square error loss can be as follows:

[0096]

[0097] Where m represents the mask time step length in real speech and n represents the mask time step length in fake speech, this minimum mean square error loss improves the difference between real speech and fake speech context.

[0098] When constructing the loss function corresponding to the model based on the two obtained loss functions, the resulting loss function can be as follows:

[0099] Loss = Loss m +Loss mse ;

[0100] Here, α can be set to be equal to 0.1.

[0101] The constructed loss function enables iterative parameter optimization of the initial pre-trained model. During optimization and iteration, the parameters of each network structure in the initial pre-trained model are optimized, including those of the convolutional, context, quantization, and contrast modules. When constructing a speech detection model based on the trained speech pre-trained model, the relevant parameters from the pre-trained model can be referenced for setting.

[0102] The structure of the initial detection model constructed is as follows: Figure 6 As shown, the initial detection model includes a convolutional module, a context module, and a linear layer. The parameters of the convolutional module and the context module in the initial detection model can be set to be consistent with the parameters of the two modules in the speech training model, so that the parameters of the linear layer can be optimized during the tuning process.

[0103] During optimization, corresponding sample speech is input, including both real and fake speech. The initial detection model is then iteratively optimized based on the detection results. Specifically, this involves: performing speech detection on both real and fake speech samples according to the initial detection model; determining an optimized loss function based on the detected sample speech results; and iterating the parameters of the initial detection model using the optimized loss function to obtain the final speech detection model.

[0104] During optimization, the sample speech used can be the speech used to train the initial pre-trained model. Each speech has its own corresponding label information, which includes real and fake information. Then, during tuning, the sample speech is detected based on the constructed initial detection, and the optimized loss function is determined based on the detection results. The parameters of the initial detection model are then iterated based on the optimized loss function.

[0105] For example, when optimizing the initial detection model, supervised learning is performed using labeled real speech and fake speech based on the pre-trained speech model. The classification task is trained during the optimization training phase, and the loss function can be the cross-entropy loss function. Then, the optimization ends based on the loss function, and a speech detection model that can be used to detect the authenticity of speech is obtained.

[0106] In the method provided by this invention, when constructing the fake speech in the sample speech pairs for training, the text information and acoustic features of the real speech samples and fake speech samples in each sample speech pair are the same. This makes the obtained speech detection model, when performing authenticity detection and judgment, not learn the pronunciation prediction task, but take distinguishing real speech and fake speech as the optimization target, extract the intrinsic information for distinguishing the authenticity of speech, and improve the accuracy and universality of speech authenticity detection.

[0107] Based on any of the above embodiments Figure 7 This is a schematic diagram of the structure of the fake voice detection device provided by the present invention, as shown below. Figure 7 As shown, the fake voice detection device 700 includes:

[0108] The voice acquisition module 701 is used to determine the voice to be detected;

[0109] The speech detection module 702 is used to input speech into the speech detection model and obtain the speech detection result output by the speech detection model. The speech detection result is one of real speech and fake speech.

[0110] The speech detection model is trained by comparing sample speech pairs, which include real speech samples and fake speech samples. The text information and acoustic features of real speech samples and fake speech samples are the same.

[0111] The apparatus provided in this invention constructs fake speech in the sample speech pairs for training, ensuring that the text information and acoustic features of the real speech samples and fake speech samples in each sample speech pair are the same. This allows the obtained speech detection model to perform authenticity detection and judgment not by learning pronunciation prediction tasks, but by using the distinction between real speech and fake speech as the optimization objective, extracting intrinsic information for distinguishing the authenticity of speech, thereby improving the accuracy and versatility of speech authenticity detection.

[0112] Based on any of the above embodiments, a model training module is further included, used for:

[0113] Determine the initial pre-trained model and sample speech pairs;

[0114] Based on the initial pre-trained model, feature extraction is performed on sample speech pairs to obtain the first feature of real speech samples and the second feature of fake speech samples.

[0115] A loss function is constructed based on the first and second features, and the parameters of the initial pre-trained model are iterated based on the loss function to obtain the speech pre-trained model.

[0116] An initial detection model is constructed based on the model parameters of the pre-trained speech model, and then the initial detection model is optimized based on sample speech pairs to obtain the speech detection model.

[0117] Based on any of the above embodiments, the model training module is further used for:

[0118] Feature extraction is performed on real speech samples to obtain the first convolutional features of real speech samples, and feature extraction is performed on fake speech samples to obtain the second convolutional features of fake speech samples.

[0119] The first convolutional feature and the second convolutional feature are masked respectively, and the first convolutional feature and the second convolutional feature after masking are extracted to obtain the first context feature and the second context feature.

[0120] The first and second convolutional features are quantized to obtain the first quantized features and the second quantized features.

[0121] The first feature is determined based on the first context feature and the first quantization feature, and the second feature is determined based on the second context feature and the second quantization feature.

[0122] Based on any of the above embodiments, the model training module is further used for:

[0123] Determine the text information corresponding to the sample speech pairs, and perform word segmentation on the text information;

[0124] Select several words from the words obtained from word segmentation to form the first word set, and determine the time step of each word in the first word set;

[0125] Mask the first and second convolutional features based on the time step.

[0126] Based on any of the above embodiments, the model training module is further used for:

[0127] The contrastive loss function is determined based on the first contextual feature, the second contextual feature, the first quantization feature, and the second quantization feature;

[0128] Determine the context feature loss function based on the first context feature and the second context feature;

[0129] The loss function of the initial pre-trained model is determined based on the contrastive loss function and the contextual feature loss function.

[0130] Based on any of the above embodiments, the model training module is further used for:

[0131] Based on the initial detection model, speech detection is performed on real speech samples and fake speech samples respectively, and the optimized loss function is determined based on the speech detection results of the samples.

[0132] The initial detection model is iterated by optimizing the loss function to obtain the speech detection model.

[0133] Based on any of the above embodiments, the forged speech sample is obtained by fusing the text information of the real speech sample with the acoustic features of the real speech sample, wherein the acoustic features include at least timbre features and voiceprint features.

[0134] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a spoofed speech detection method. This method includes: determining the speech to be detected; inputting the speech into a speech detection model to obtain a speech detection result output by the speech detection model, wherein the speech detection result is either real speech or spoofed speech; wherein the speech detection model is obtained by comparative training based on sample speech pairs, the sample speech pairs including real speech samples and spoofed speech samples, and the text information and acoustic features of the real speech samples and spoofed speech samples are identical.

[0135] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the forged speech detection method provided by the above methods, determine the speech to be detected; input the speech into a speech detection model, and obtain the speech detection result output by the speech detection model, wherein the speech detection result is one of real speech and forged speech; wherein the speech detection model is obtained by comparative training based on sample speech pairs, the sample speech pairs including real speech samples and forged speech samples, and the text information and acoustic features of the real speech samples and the forged speech samples are the same.

[0137] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the aforementioned methods for detecting spoofed speech. The method includes: determining the speech to be detected; inputting the speech into a speech detection model to obtain a speech detection result output by the speech detection model, wherein the speech detection result is one of real speech and spoofed speech; wherein the speech detection model is obtained by comparative training based on sample speech pairs, the sample speech pairs including real speech samples and spoofed speech samples, and the text information and acoustic features of the real speech samples and spoofed speech samples are the same.

[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of detecting a forged speech, characterized by, include: Identify the speech to be detected; The speech is input into the speech detection model to obtain the speech detection result output by the speech detection model. The speech detection result is either real speech or fake speech. The speech detection model is obtained by comparative training based on sample speech pairs, which include real speech samples and fake speech samples. The real speech samples and the fake speech samples have the same text information and acoustic features. The training steps of the speech detection model include: Determine the initial pre-trained model and sample speech pairs; Based on the initial pre-trained model, feature extraction is performed on the sample speech pairs to obtain the first feature of the real speech sample and the second feature of the fake speech sample; A loss function is constructed based on the first feature and the second feature, and the parameters of the initial pre-trained model are iterated based on the loss function to obtain a speech pre-trained model; An initial detection model is constructed based on the model parameters of the pre-trained speech model, and the initial detection model is optimized based on the sample speech pairs to obtain a speech detection model.

2. The counterfeit speech detection method of claim 1, wherein, The step of extracting features from the sample speech pairs to obtain the first feature of the real speech sample and the second feature of the fake speech sample includes: Feature extraction is performed on the real speech sample to obtain the first convolutional feature of the real speech sample, and feature extraction is performed on the fake speech sample to obtain the second convolutional feature of the fake speech sample; The first convolutional feature and the second convolutional feature are masked respectively, and the first convolutional feature and the second convolutional feature after masking are extracted to obtain the first context feature and the second context feature. The first convolutional feature and the second convolutional feature are quantized to obtain the first quantized feature and the second quantized feature; The first feature is determined based on the first context feature and the first quantization feature, and the second feature is determined based on the second context feature and the second quantization feature.

3. The method of detecting a forged voice according to claim 2, wherein The masking of the first convolutional feature and the second convolutional feature includes: Determine the text information corresponding to the sample speech pair, and perform word segmentation on the text information; Select several words from the words obtained from word segmentation to form a first word set, and determine the time step of each word in the first word set; Masking is performed on the first convolutional feature and the second convolutional feature according to the time step.

4. The method of detecting a forged voice according to claim 2, wherein The step of constructing a loss function based on the first feature and the second feature includes: The contrast loss function is determined based on the first context feature, the second context feature, the first quantization feature, and the second quantization feature; Determine the context feature loss function based on the first context feature and the second context feature; The loss function of the initial pre-trained model is determined based on the contrast loss function and the context feature loss function.

5. The counterfeit speech detection method of claim 1, wherein, The step of optimizing the initial detection model based on the sample speech to obtain a speech detection model includes: Based on the initial detection model, speech detection is performed on the real speech samples and the fake speech samples respectively, and an optimized loss function is determined based on the obtained sample speech detection results; The initial detection model is iterated based on the optimized loss function to obtain the speech detection model.

6. The method of detecting a forged voice according to any one of claims 1 to 5, wherein, The forged speech sample is obtained by fusing the text information of the real speech sample with the acoustic features of the real speech sample, wherein the acoustic features include at least timbre features and voiceprint features.

7. A forged speech detection apparatus characterized by comprising: include: The speech acquisition module is used to determine the speech to be detected; The speech detection module is used to input the speech into the speech detection model and obtain the speech detection result output by the speech detection model. The speech detection result is one of real speech and fake speech. The speech detection model is obtained by comparative training based on sample speech pairs, which include real speech samples and fake speech samples. The real speech samples and the fake speech samples have the same text information and acoustic features. The training steps of the speech detection model include: Determine the initial pre-trained model and sample speech pairs; Based on the initial pre-trained model, feature extraction is performed on the sample speech pairs to obtain the first feature of the real speech sample and the second feature of the fake speech sample; A loss function is constructed based on the first feature and the second feature, and the parameters of the initial pre-trained model are iterated based on the loss function to obtain a speech pre-trained model; An initial detection model is constructed based on the model parameters of the pre-trained speech model, and the initial detection model is optimized based on the sample speech pairs to obtain a speech detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the fake voice detection method as described in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the fake voice detection method as described in any one of claims 1 to 6.

Images