Synthetic speech detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a synthetic speech detection method and device, a storage medium and electronic equipment. The synthetic speech detection method provided by the invention comprises the following steps: acquiring a to-be-detected audio input by a user; inputting the to-be-detected audio into a pre-trained detection model, wherein the detection model at least comprises a first extraction subnet, a second extraction subnet and an output subnet; extracting audio features of the to-be-detected audio through the first extraction subnet, and extracting frequency cepstrum features of the to-be-detected audio through the second extraction subnet; and outputting a detection result of the to-be-detected audio according to the audio features and the frequency cepstrum features by adopting the output subnet.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a synthetic speech detection method, device, storage medium and electronic device. Background Art

[0002] Nowadays, with the continuous development of computer and network security technology, voice authentication technology has gradually matured and has been applied in many scenarios. As a means of authentication for security protection, it can effectively prevent others from maliciously impersonating users by recognizing the user's input voice to determine the user's identity.

[0003] However, with the continuous advancement of artificial intelligence (AI) technology, voice authentication as a security protection technology is facing a challenge. Through generative technology, AI can autonomously learn and generate new content using various data, including but not limited to text, images, audio and other content. Using generative technology, attackers may impersonate users by synthesizing forged voices that are highly similar to the user's own voice, and current voice authentication technology is difficult to detect such attacks. In the end, users may be impersonated by others, which may infringe on the interests of users and leak their privacy.

[0004] Therefore, how to accurately and effectively detect forged synthesized speech impersonating users is an urgent problem to be solved. Summary of the invention

[0005] The present specification provides a synthetic speech detection method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions: This specification provides a synthetic speech detection method, comprising: Get the audio input by the user to be detected; Inputting the audio to be detected into a pre-trained detection model, wherein the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; Extracting audio features of the audio to be detected by the first extraction subnet, and extracting frequency cepstrum features of the audio to be detected by the second extraction subnet; The output subnet is used to output the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

[0007] Optionally, the first extraction subnetwork includes at least a plurality of first extraction layers and a splicing layer; Extracting the audio features of the audio to be detected by the first extraction subnet specifically includes: Inputting the audio to be detected into the first extraction subnet to obtain intermediate features output by each first extraction layer included in the first extraction subnet; The intermediate features output by the first extraction layers are spliced ​​through the splicing layer to obtain the audio features output by the first extraction subnet.

[0008] Optionally, the first extraction subnetwork includes at least a second extraction layer and a plurality of difference layers; Extracting the frequency cepstrum feature of the audio to be detected by the second extraction subnet specifically includes: Inputting the audio to be detected into the second extraction layer of the second extraction subnet to obtain an original feature sequence of the audio to be detected extracted by the second extraction layer; Inputting the original feature sequence into each difference layer included in the second extraction subnet to obtain a difference feature sequence output by each difference layer; The differential feature sequence is concatenated with the original feature sequence to obtain the frequency cepstrum feature output by the second extraction subnet.

[0009] Optionally, inputting the original feature sequence into each difference layer included in the second extraction subnet to obtain the difference feature sequence output by each difference layer specifically includes: Inputting the original feature sequence into the difference layer; For each original feature included in the original feature sequence, determine a differential feature corresponding to the original feature according to the original feature and the original features adjacent to the original feature in the original feature sequence; A differential feature sequence is constructed according to the determined differential features.

[0010] Optionally, outputting a detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature specifically includes: Fusing the audio feature with the frequency cepstrum feature to obtain a fused feature; According to the fusion feature, the detection result of the audio to be detected is output.

[0011] Optionally, pre-train a classification model, specifically including: Obtaining sample audio and a labeling result indicating whether the sample audio is synthesized audio; Inputting the sample audio into a classification model to be trained; Extracting the audio features to be optimized of the sample audio through the first extraction subnet, and extracting the frequency cepstrum features to be optimized of the sample audio through the second extraction subnet; Using the output subnet, outputting the to-be-optimized detection result of the sample audio according to the to-be-optimized audio feature and the to-be-optimized frequency cepstrum feature; The classification model is trained according to the difference between the detection result to be optimized and the annotation result.

[0012] This specification provides a synthetic speech detection device, comprising: An acquisition module, used to acquire the audio to be detected input by the user; An input module, used for inputting the audio to be detected into a pre-trained detection model, wherein the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; An extraction module, configured to extract the audio features of the audio to be detected through the first extraction subnet, and to extract the frequency cepstrum features of the audio to be detected through the second extraction subnet; The output module is used to use the output subnet to output the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

[0013] Optionally, the first extraction subnetwork includes at least a plurality of first extraction layers and a splicing layer; The extraction module is specifically used to input the audio to be detected into the first extraction subnet to obtain the intermediate features output by the first extraction layers contained in the first extraction subnet; and splice the intermediate features output by the first extraction layers through the splicing layer to obtain the audio features output by the first extraction subnet.

[0014] Optionally, the first extraction subnetwork includes at least a second extraction layer and a plurality of difference layers; The extraction module is specifically used to input the audio to be detected into the second extraction layer of the second extraction subnet to obtain the original feature sequence of the audio to be detected extracted by the second extraction layer; input the original feature sequence into each difference layer included in the second extraction subnet to obtain the differential feature sequence output by each difference layer; splice the differential feature sequence with the original feature sequence to obtain the frequency cepstrum feature output by the second extraction subnet.

[0015] Optionally, the extraction module is specifically used to input the original feature sequence into the differential layer; for each original feature contained in the original feature sequence, determine the differential feature corresponding to the original feature based on the original feature and the original features adjacent to the original feature in the original feature sequence; and construct a differential feature sequence based on the determined differential features.

[0016] Optionally, the output module is specifically used to fuse the audio feature with the frequency cepstrum feature to obtain a fused feature; and output a detection result of the audio to be detected based on the fused feature.

[0017] Optionally, the device also includes a training module, which is specifically used to obtain sample audio and a labeling result indicating whether the sample audio is a synthetic audio; input the sample audio into a classification model to be trained; extract the audio features to be optimized of the sample audio through the first extraction subnet, and extract the frequency cepstrum features to be optimized of the sample audio through the second extraction subnet; use the output subnet to output the detection result to be optimized of the sample audio based on the audio features to be optimized and the frequency cepstrum features to be optimized; and train the classification model based on the difference between the detection result to be optimized and the labeling result.

[0018] The present specification provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned synthetic speech detection method is implemented.

[0019] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned synthetic speech detection method when executing the program.

[0020] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: In the synthetic speech detection method provided in the present specification, the audio to be detected input by the user is obtained; the audio to be detected is input into a pre-trained detection model, and the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; the audio features of the audio to be detected are extracted by the first extraction subnet, and the frequency cepstrum features of the audio to be detected are extracted by the second extraction subnet; and the output subnet is used to output the detection result of the audio to be detected according to the audio features and the frequency cepstrum features.

[0021] When the synthetic speech detection method provided in this specification is used to detect whether the audio to be detected is synthetic audio, the audio features and frequency cepstrum features of the audio to be detected can be extracted respectively by the first extraction subnet and the second extraction subnet in the detection model, and the final detection result can be outputted by the output subnet according to the extracted audio features and frequency cepstrum features. This method combines audio features with frequency cepstrum features, and on the basis of analyzing the pitch, timbre and other contents of the audio to be detected, additionally considers the frequency, frequency change rate and acceleration of the audio to be detected, and can more accurately detect whether the output audio to be detected is synthetic audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a synthetic speech detection method provided in this specification; Figure 2 A schematic diagram of the model structure of a detection model provided in this specification; Figure 3 A schematic diagram of a synthetic speech detection device provided in this specification; Figure 4 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0024] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 A flow chart of a synthetic speech detection method provided in this specification includes the following steps: S100: Acquire the audio to be detected input by the user.

[0026] In this specification, the execution entity used to implement the synthetic speech detection method may refer to a designated device such as a server set up on the business platform. For the sake of ease of description, this specification only takes the server as the execution entity as an example to illustrate a synthetic speech detection method provided in this specification.

[0027] This method is mainly used in the voice verification link when verifying the identity of the user, and the identity of the user is determined by recognizing the voice input by the user. Based on this, in this step, the audio to be detected input by the user can be first obtained. Among them, the audio to be detected input by the user can be a pre-recorded audio file, or it can be a voice clip directly recorded by a recording device such as a microphone, and this manual does not make specific restrictions on this.

[0028] This method can be applied to a variety of scenarios where user identity verification is required, such as unlocking electronic devices, authorizing software functions, interacting with artificial intelligence, etc. This method can accurately determine whether the audio to be detected is a fake user voice synthesized by generative technology based on the audio to be detected input by the user.

[0029] S102: Input the audio to be detected into a pre-trained detection model, where the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet.

[0030] In this method, a pre-trained detection model is used to complete the detection of the audio to be detected. The function of the detection model is to determine whether the audio to be detected is a synthetic audio based on the input audio to be detected. Figure 2 A specific embodiment of a model structure of an adoptable detection model is given.

[0031] like Figure 2 As shown, the detection model used in the synthetic speech detection method provided in this specification may include at least a first extraction subnet, a second extraction subnet and an output subnet. The first extraction subnet and the second extraction subnet may be used to extract two different features of the input audio to be detected respectively; the output subnet may detect the audio to be detected based on the two different features extracted by the first extraction subnet and the second extraction subnet, and output a detection result of whether it is a forged synthetic speech.

[0032] certainly, Figure 2 The model structure of the detection model given is only one of many optional specific embodiments. Figure 2 In addition to the subnets included in the detection model shown, the detection model may also include other subnets to enhance or expand the functionality of the detection model, and this specification does not impose any specific restrictions on this.

[0033] S104: extracting audio features of the audio to be detected through the first extraction subnet, and extracting frequency cepstrum features of the audio to be detected through the second extraction subnet.

[0034] In the audio detection method provided in this specification, there are two subnets in the detection model for extracting features of the audio to be detected, namely the first extraction subnet and the second extraction subnet. The structures and parameters of the two are different. This specification will introduce the two subnets separately.

[0035] The function of the first extraction subnet is to extract the audio features of the audio to be detected. The first extraction subnet includes at least a number of first extraction layers and a splicing layer. When the first extraction subnet is used to extract the audio features of the audio to be detected, the audio to be detected can be specifically input into the first extraction subnet to obtain the intermediate features output by the first extraction layers contained in the first extraction subnet; the intermediate features output by the first extraction layers are spliced ​​through the splicing layer to obtain the audio features output by the first extraction subnet.

[0036] like Figure 2 As shown, the first extraction subnet may include several first extraction layers, and each first extraction layer may output an intermediate feature. The number of first extraction layers can be set according to specific needs. Generally, the more first extraction layers there are, the better the effect of the audio features output by the final first extraction subnet, but correspondingly, the model needs to process a larger amount of data, and the output efficiency will be reduced. Under the condition that the first extraction layer can extract the intermediate features, the structure of the first extraction layer can adopt a variety of different forms. At the same time, the structures and parameters of different first extraction layers can be the same or different, and this manual does not impose specific restrictions on this.

[0037] Figure 2 A feasible structural diagram of the first extraction subnet is given, as follows: Figure 2 As shown, in this example, the first extraction subnet includes 4 first extraction layers, which exist in the first extraction layer subnet in a "series" form. The input of the first first extraction layer is the audio to be detected, and the input of each subsequent first extraction layer is the output of the previous first extraction layer, or the output of the previous first extraction layer and the audio to be detected.

[0038] In a specific embodiment, the Whisper network can be used as the first extraction subnet. The Whisper network is a deep learning model for automatic speech recognition (ASR). The core of the Whisper network structure is the Transformer architecture, which is composed of multiple identical layers stacked. Each layer includes a multi-head self-attention mechanism and a feed-forward neural network. When used as the first extraction subnet, the minimum version of the Whisper network can be selected, which includes 4 first extraction layers. The features extracted by the first extraction subnet are represented as Z, and its dimension is (B, L, T, dim). Among them, B represents the batch size; L represents the number of first extraction layers, such as 4 used in the specific embodiment; T represents the length of the speech sample, which is usually a fixed value, such as 5s, 10s, 20s, 30s, etc.; dim represents the dimension of each layer of features, and its value is 384.

[0039] It is worth mentioning that the first extraction layers can be related to each other and run in a progressive relationship, or they can be run separately. When the first extraction layers run separately, the input of each first extraction layer is the audio to be detected. Of course, no matter which method is used, the intermediate features output by each first extraction layer can be spliced ​​into audio features through the splicing layer.

[0040] The function of the second extraction subnet is to extract the frequency cepstrum features of the audio to be detected. In this method, the frequency cepstrum features of the audio to be detected are extracted based on the Mel Frequency Cepstrum Coefficient (MFCC). MFCC is a feature extracted from the spectrum of the audio signal using the nonlinear correspondence between the auditory characteristics of the human ear (Mel frequency) and the frequency (Hz). In layman's terms, the Mel frequency is based on the auditory characteristics of the human ear, and it has a nonlinear correspondence with the general frequency. The human ear's perception of different frequencies is not linear, and MFCC is designed based on this characteristic. Through MFCC, key spectral features can be extracted from the speech signal, thereby performing tasks such as speech detection, classification or retrieval.

[0041] In the present method, the second extraction subnet may include at least a second extraction layer and a plurality of difference layers. When the second extraction subnet is used to extract the frequency cepstrum features of the audio to be detected, specifically, the audio to be detected is input into the second extraction layer of the second extraction subnet to obtain the original feature sequence of the audio to be detected extracted by the second extraction layer; the original feature sequence is input into each difference layer included in the second extraction subnet to obtain the difference feature sequence output by each difference layer; the difference feature sequence is spliced ​​with the original feature sequence to obtain the frequency cepstrum features output by the second extraction subnet.

[0042] Figure 2 A feasible structural diagram of the second extraction subnet is given, as follows: Figure 2 As shown, in this example, the second extraction subnetwork includes a second extraction layer and two difference layers. In the second extraction subnetwork of the detection model adopted by this method, the second extraction layer is used to extract the original Mel frequency cepstral coefficients, that is, the original features of the audio to be detected; the difference layer is used to output differential features based on the original features extracted by the second extraction layer.

[0043] It should be noted that when the second extraction subnet extracts the frequency cepstrum features of the audio to be detected, the second extraction layer will extract the audio to be detected frame by frame. In other words, for each frame of audio contained in the audio to be detected, the second extraction layer will extract a corresponding original feature. After sorting the original features corresponding to each frame of audio extracted according to the arrangement order of each frame of audio in the audio to be detected, the original features of the entire audio to be detected, that is, the original feature sequence, can be obtained.

[0044] Similarly, when the differential layer performs a differential operation on the original feature sequence of the audio to be detected, the processing is also performed frame by frame. Specifically, the original feature sequence can be input into the differential layer; for each original feature contained in the original feature sequence, the differential feature corresponding to the original feature is determined according to the original feature and the original features adjacent to the original feature in the original feature sequence; and the differential feature sequence is formed according to the determined differential features.

[0045] The relationship between the difference layers contained in the second extraction subnetwork is a layer-by-layer progressive relationship. In other words, according to the arrangement order of the difference layers in the second extraction subnetwork, their outputs are respectively a first-order difference feature sequence, a second-order difference feature sequence, and so on, where the original feature sequence can be regarded as a zero-order difference feature sequence.

[0046] In this method, the differential features of the audio to be detected are used to characterize the derivative of the Mel-frequency cepstral coefficients over time. Specifically, it can be calculated according to the following formula:

[0047] Among them, C t Represents the current order feature of the audio frame corresponding to time t, ΔC t represents the next-order feature of the audio of the frame corresponding to time t after derivation. For example, when the feature to be calculated is a first-order differential feature, then C t is the original feature, ΔC t is a first-order differential feature; when the feature to be calculated is a second-order differential feature, then C t is the first-order difference characteristic, ΔC t is the second-order differential feature. N represents the number of adjacent frames used to calculate the differential features of a frame of audio, which can usually be 1 or 2. For each frame of audio contained in the audio to be detected, the above formula can be used to determine its differential features of each order. The differential features of each frame of audio at the same order are arranged according to the order of each frame of audio in the audio to be detected, and the differential feature sequence of the order corresponding to the audio to be detected can be obtained.

[0048] After obtaining the original feature sequence output by the second extraction layer and the differential feature sequence output by each differential layer, the outputs of each layer in the second extraction subnet can be fused to obtain the frequency cepstrum feature of the final output of the second extraction subnet. Specifically, it can be expressed as:

[0049] Among them, X f represents the fused frequency cepstrum feature, X i represents the i-order differential feature. When i=0, X i Represents the original feature.

[0050] Furthermore, in the second extraction subnetwork of the detection model used in the present method, the number of difference layers can be set according to specific needs. In general, the first-order difference features and the second-order difference features respectively provide information about the rate and acceleration of change of the speech signal of the audio to be detected. These dynamic features can effectively help the detection model capture the timing information such as the rhythm, stress and consonant explosion of the audio to be detected, and can provide key assistance in distinguishing natural speech from synthesized speech. Therefore, a two-layer difference layer design can generally be adopted in the second extraction subnetwork, that is, extracting the first-order difference feature sequence and the second-order difference feature sequence.

[0051] S106: Using the output subnet, outputting the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

[0052] Finally, the output subnet in the detection model can judge the audio to be detected based on the audio features output by the first extraction subnet and the frequency cepstrum features output by the second extraction subnet, and output a detection result of whether it is synthetic audio.

[0053] Additionally, when the output subnet outputs the detection result based on the audio feature and the frequency cepstrum feature, in addition to directly outputting the result, the two can also be fused before outputting the result. Specifically, the audio feature and the frequency cepstrum feature can be fused to obtain a fused feature; and the detection result of the audio to be detected is output based on the fused feature.

[0054] The network structure of the output subnet can exist in many different forms. For example, a binary classification network can be used as the output subnet. This specification does not impose any specific restrictions on this. It only needs to ensure that the output subnet can output the detection results based on the input audio features and cepstrum frequency features.

[0055] When the synthetic speech detection method provided in this specification is used to detect whether the audio to be detected is synthetic audio, the audio features and frequency cepstrum features of the audio to be detected can be extracted respectively by the first extraction subnet and the second extraction subnet in the detection model, and the final detection result can be outputted by the output subnet according to the extracted audio features and frequency cepstrum features. This method combines audio features with frequency cepstrum features, and on the basis of analyzing the pitch, timbre and other contents of the audio to be detected, additionally considers the frequency, frequency change rate and acceleration of the audio to be detected, and can more accurately detect whether the output audio to be detected is synthetic audio.

[0056] Furthermore, the detection model used in the synthetic speech detection method provided in this specification can be pre-trained. Specifically, sample audio and a labeling result indicating whether the sample audio is synthetic audio can be obtained; the sample audio is input into the classification model to be trained; the audio features to be optimized of the sample audio are extracted through the first extraction subnet, and the frequency cepstrum features to be optimized of the sample audio are extracted through the second extraction subnet; the output subnet is used to output the detection result to be optimized of the sample audio according to the audio features to be optimized and the frequency cepstrum features to be optimized; the classification model is trained according to the difference between the detection result to be optimized and the labeling result.

[0057] During training, audio that can clearly determine whether it is synthetic audio can be directly used as sample audio, and the result of whether the sample audio is synthetic audio can be used as the annotation result. The annotation result is usually the same as the content that the detection model can output, such as "yes" and "no", or "1" and "0", etc. Among them, "yes" means that the sample audio is actually synthetic audio, and "no" means that the sample audio is not actually synthetic audio. After the sample audio is input into the detection model to be trained, the detection model can, through the operation of the first extraction subnet, the second extraction subnet and the output subnet, ultimately give an inaccurate detection result, that is, the detection result to be optimized. The result output by the detection model uses the difference between the detection result to be optimized and the annotated detection result as the loss during model optimization, adjusts the various parameters in each subnet in the detection model, and finally completes the training of the detection model.

[0058] The above are one or more methods for implementing synthetic speech detection in this specification. Based on the same idea, this specification also provides a corresponding synthetic speech detection device, such as Figure 3 shown.

[0059] Figure 3 A schematic diagram of a synthetic speech detection device provided in this specification includes: The acquisition module 200 is used to acquire the audio to be detected input by the user; An input module 202, configured to input the audio to be detected into a pre-trained detection model, wherein the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; An extraction module 204, configured to extract the audio features of the audio to be detected through the first extraction subnet, and to extract the frequency cepstrum features of the audio to be detected through the second extraction subnet; The output module 206 is used to use the output subnet to output the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

[0060] Optionally, the first extraction subnetwork includes at least a plurality of first extraction layers and a splicing layer; The extraction module 204 is specifically used to input the audio to be detected into the first extraction subnet to obtain the intermediate features output by the first extraction layers contained in the first extraction subnet; and to splice the intermediate features output by the first extraction layers through the splicing layer to obtain the audio features output by the first extraction subnet.

[0061] Optionally, the first extraction subnetwork includes at least a second extraction layer and a plurality of difference layers; The extraction module 204 is specifically used to input the audio to be detected into the second extraction layer of the second extraction subnet to obtain the original feature sequence of the audio to be detected extracted by the second extraction layer; input the original feature sequence into each difference layer included in the second extraction subnet to obtain the differential feature sequence output by each difference layer; splice the differential feature sequence with the original feature sequence to obtain the frequency cepstrum feature output by the second extraction subnet.

[0062] Optionally, the extraction module 204 is specifically used to input the original feature sequence into the differential layer; for each original feature included in the original feature sequence, determine the differential feature corresponding to the original feature based on the original feature and the original features adjacent to the original feature in the original feature sequence; and construct a differential feature sequence based on the determined differential features.

[0063] Optionally, the output module 206 is specifically configured to fuse the audio feature with the frequency cepstrum feature to obtain a fused feature; and output a detection result of the audio to be detected according to the fused feature.

[0064] Optionally, the device also includes a training module 208, which is specifically used to obtain sample audio and a labeling result indicating whether the sample audio is a synthetic audio; input the sample audio into a classification model to be trained; extract the audio features to be optimized of the sample audio through the first extraction subnet, and extract the frequency cepstrum features to be optimized of the sample audio through the second extraction subnet; use the output subnet to output the detection result to be optimized of the sample audio according to the audio features to be optimized and the frequency cepstrum features to be optimized; and train the classification model according to the difference between the detection result to be optimized and the labeling result.

[0065] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A synthetic speech detection method is provided.

[0066] This manual also provides Figure 4 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 4 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0067] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0068] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0069] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0070] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0071] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0076] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0077] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0078] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0079] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0081] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0082] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A synthetic speech detection method, characterized in that: include: Get the audio input by the user to be detected; Inputting the audio to be detected into a pre-trained detection model, wherein the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; Extracting audio features of the audio to be detected by the first extraction subnet, and extracting frequency cepstrum features of the audio to be detected by the second extraction subnet; The output subnet is used to output the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

2. The method according to claim 1, characterized in that The first extraction subnetwork at least includes a plurality of first extraction layers and a splicing layer; Extracting the audio features of the audio to be detected by the first extraction subnet specifically includes: Inputting the audio to be detected into the first extraction subnet to obtain intermediate features output by the first extraction layers included in the first extraction subnet; The intermediate features output by the first extraction layers are spliced ​​through the splicing layer to obtain the audio features output by the first extraction subnet.

3. The method according to claim 1, characterized in that The first extraction subnetwork at least includes a second extraction layer and a plurality of difference layers; Extracting the frequency cepstrum feature of the audio to be detected by the second extraction subnet specifically includes: Inputting the audio to be detected into the second extraction layer of the second extraction subnet to obtain an original feature sequence of the audio to be detected extracted by the second extraction layer; Inputting the original feature sequence into each difference layer included in the second extraction subnet to obtain a difference feature sequence output by each difference layer; The differential feature sequence is concatenated with the original feature sequence to obtain the frequency cepstrum feature output by the second extraction subnet.

4. The method according to claim 3, characterized in that Inputting the original feature sequence into each difference layer included in the second extraction subnet to obtain the difference feature sequence output by each difference layer specifically includes: Inputting the original feature sequence into the difference layer; For each original feature included in the original feature sequence, determine a differential feature corresponding to the original feature according to the original feature and the original features adjacent to the original feature in the original feature sequence; A differential feature sequence is constructed according to the determined differential features.

5. The method according to claim 1, characterized in that According to the audio feature and the frequency cepstrum feature, Outputting the detection result of the audio to be detected specifically includes: Fusing the audio feature with the frequency cepstrum feature to obtain a fused feature; According to the fusion feature, the detection result of the audio to be detected is output.

6. The method according to claim 1, characterized in that Pre-trained classification models, including: Obtaining sample audio and a labeling result indicating whether the sample audio is synthesized audio; Inputting the sample audio into a classification model to be trained; Extracting the audio features to be optimized of the sample audio through the first extraction subnet, and extracting the frequency cepstrum features to be optimized of the sample audio through the second extraction subnet; Using the output subnet, outputting the to-be-optimized detection result of the sample audio according to the to-be-optimized audio feature and the to-be-optimized frequency cepstrum feature; The classification model is trained according to the difference between the detection result to be optimized and the annotation result.

7. A synthetic speech detection device, characterized in that: include: An acquisition module is used to acquire the audio to be detected input by the user; An input module, used for inputting the audio to be detected into a pre-trained detection model, wherein the detection model at least includes a first extraction subnet, a second extraction subnet, and an output subnet; An extraction module, configured to extract the audio features of the audio to be detected through the first extraction subnet, and to extract the frequency cepstrum features of the audio to be detected through the second extraction subnet; The output module is used to use the output subnet to output the detection result of the audio to be detected according to the audio feature and the frequency cepstrum feature.

8. The device according to claim 7, characterized in that The first extraction subnetwork at least includes a plurality of first extraction layers and a splicing layer; The extraction module is specifically used to input the audio to be detected into the first extraction subnet to obtain the intermediate features output by the first extraction layers contained in the first extraction subnet; and splice the intermediate features output by the first extraction layers through the splicing layer to obtain the audio features output by the first extraction subnet.

9. The device according to claim 7, characterized in that The first extraction subnetwork at least includes a second extraction layer and a plurality of difference layers; The extraction module is specifically used to input the audio to be detected into the second extraction layer of the second extraction subnet to obtain the original feature sequence of the audio to be detected extracted by the second extraction layer; input the original feature sequence into each difference layer included in the second extraction subnet to obtain the difference feature sequence output by each difference layer; The differential feature sequence is concatenated with the original feature sequence to obtain the frequency cepstrum feature output by the second extraction subnet.

10. The device according to claim 9, characterized in that The extraction module is specifically used to input the original feature sequence into the differential layer; for each original feature contained in the original feature sequence, determine the differential feature corresponding to the original feature according to the original feature and the original features adjacent to the original feature in the original feature sequence; and form a differential feature sequence according to the determined differential features.

11. The device according to claim 7, characterized in that The output module is specifically used to fuse the audio feature with the frequency cepstrum feature to obtain a fused feature; and output the detection result of the audio to be detected according to the fused feature.

12. The device according to claim 7, characterized in that The device also includes a training module, which is specifically used to obtain sample audio and a labeling result indicating whether the sample audio is a synthetic audio; input the sample audio into a classification model to be trained; extract the audio features to be optimized of the sample audio through the first extraction subnet, and extract the frequency cepstrum features to be optimized of the sample audio through the second extraction subnet; The output subnet is used to output the detection result to be optimized of the sample audio according to the audio feature to be optimized and the frequency cepstrum feature to be optimized; and the classification model is trained according to the difference between the detection result to be optimized and the labeling result.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

14. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images