A human voice detection method, apparatus, device and medium

Abstract

The application belongs to the technical field of human voice anti-counterfeiting identification, and discloses a human voice detection method, device, equipment and medium, the method comprises the following steps: obtaining a speech data set; setting a first adaptive module and a second adaptive module in a transformer module of a pre-trained speech model, obtaining an adaptive pre-trained speech detection model; setting a frame level information entropy detection module at an output end of the adaptive pre-trained speech detection model, obtaining a human voice detection model; training the human voice detection model by using the speech data set and a joint loss function, and optimizing the human voice detection model by using an Adam optimization module, obtaining a target human voice detection model; and identifying and judging input speech by using the target human voice detection model to determine whether the input speech is counterfeit speech. The application improves the generalization of the human voice recognition detection model, and the joint loss function is used to train the human voice detection model, so that the target human voice detection model has stronger robustness.

Description

Technical Field

[0001] This invention relates to the field of voice anti-counterfeiting recognition technology, and in particular to a voice detection method, device, equipment and medium. Background Technology

[0002] Voice synthesis technology uses advanced speech processing techniques to create or modify audio content to sound as if it were spoken by a specific individual, even if that individual never made those statements or if their tone and intonation differ. This technology has broad applications; while it can be used to create entertainment content, it could also be used to spread rumors or commit fraud.

[0003] The rapid development of speech synthesis and conversion technologies has spurred the advancement of voice spoofing techniques. On the one hand, this poses a new challenge to traditional voiceprint recognition technology; on the other hand, music generation technologies also present new challenges to traditional music copyright protection. Therefore, developing detection technologies for deepfakes of synthesized voices has become crucial.

[0004] Currently, deepfake detection techniques for synthetic voices mainly fall into two categories: The first category employs a two-step approach combining feature extraction and a classifier. First, the voice sample undergoes feature analysis, and then a classification model is trained based on these features to determine whether the voice is fake. This method is relatively cumbersome to implement, its classification accuracy depends on the quality of feature extraction, and the two-step approach may result in unsatisfactory anti-spoofing effects. The second category directly trains an end-to-end classification model to detect fake speech, but this method requires a large amount of training data and computational power to achieve good performance. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, device, and medium for human voice detection to solve the problems of cumbersome implementation and poor detection and recognition effect of current deepfake human voice detection technology.

[0006] To address the aforementioned technical problems, this application provides a method for human voice detection, employing the technical solution described below, including:

[0007] Acquire a speech dataset and process the speech dataset to obtain a target speech frame sequence. The speech dataset includes fake speech and the corresponding real speech.

[0008] A first adaptive module and a second adaptive module are set between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model;

[0009] A frame-level information entropy detection module is set at the output of the adaptive pre-trained speech detection model to obtain a human voice detection model.

[0010] Obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model;

[0011] The target speech frame sequence is input into the target human voice detection model. The frame level information entropy detection module is used to obtain the frame level entropy value, and the frame level entropy value is used to determine whether the speech data is fake speech. The frame level entropy value is Gaussian distributed.

[0012] Furthermore, the step of processing the speech dataset to obtain the target speech frame sequence includes:

[0013] The speech in the speech dataset is sampled according to a preset sampling rate to obtain the speech frame sequence;

[0014] The speech frame sequence is normalized to obtain a target speech frame sequence of a specified length.

[0015] Furthermore, the step of setting a first adaptive module and a second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model includes:

[0016] Connect the output of the second fully connected layer to the input of the first residual layer, and connect the output of the first residual layer to the input of the third fully connected layer to obtain the first adaptive module;

[0017] Connect the first adaptive module between the multi-head attention layer and the first normalization layer;

[0018] Connect the output of the fourth fully connected layer to the input of the second residual layer, and connect the output of the second residual layer to the input of the fifth fully connected layer to obtain the second adaptive module;

[0019] The second adaptive module is connected between the first fully connected layer and the second normalized layer to generate the adaptive pre-trained speech detection model.

[0020] Furthermore, obtaining the joint loss function includes:

[0021] The cross-entropy between the real speech category and the predicted speech category is calculated using the predicted probability distribution of the target human voice detection model to obtain the classification loss function;

[0022] Calculate the first MEL spectrum of the predicted speech and the second MEL spectrum of the real speech, calculate the mean square error of the first MEL spectrum and the second MEL spectrum, and obtain the reconstruction loss function.

[0023] Obtain the mean and variance of the Gaussian distribution of the frame horizontal entropy value, and calculate the KL loss function based on the mean and variance;

[0024] The classification loss function, the reconstruction loss function, and the KL loss function are added together to obtain the joint loss function.

[0025] Furthermore, the step of inputting the target speech frame sequence into the target human voice detection model, obtaining the frame-level entropy value using the frame-level information entropy detection module, and using the frame-level entropy value to determine whether the speech data is fake speech, wherein the frame-level entropy value is a Gaussian distribution, includes:

[0026] The target speech frame sequence is input into the adaptive pre-trained speech detection model to obtain the corresponding embedding vector;

[0027] The embedding vector is input into the frame horizontal information entropy detection module to obtain the frame horizontal entropy value corresponding to the embedding vector;

[0028] The frame horizontal entropy value is compared with a preset threshold.

[0029] If the frame horizontal entropy value is greater than the preset threshold, it is determined to be real speech;

[0030] If the frame horizontal entropy value is less than or equal to the preset threshold, it is determined to be fake speech.

[0031] Furthermore, the step of inputting the embedding vector into the frame horizontal information entropy detection module to obtain the frame horizontal entropy value corresponding to the embedding vector includes:

[0032] The embedding vectors are input into the mean encoder and the variance encoder respectively to obtain the corresponding frame level Gaussian distribution.

[0033] Calculate the entropy of the horizontal Gaussian distribution of the frame to obtain the horizontal entropy value of the first frame;

[0034] The covariance matrix of the Gaussian distribution of the horizontal entropy values ​​in the first frame is a diagonal matrix.

[0035] To address the aforementioned issues, a human voice detection device is also provided, employing the aforementioned human voice detection method based on frame-level information entropy, comprising:

[0036] The first acquisition module is used to acquire a voice dataset and process the voice dataset to acquire a target voice frame sequence. The voice dataset includes fake voice and corresponding real voice.

[0037] The second acquisition module is used to set the first adaptive module and the second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model.

[0038] The third acquisition module is used to set a frame-level information entropy detection module at the output end of the adaptive pre-trained speech detection model to obtain a human voice detection model.

[0039] The training module is used to obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model.

[0040] The decision module is used to input the target speech frame sequence into the target human voice detection model, obtain the frame level entropy value using the frame level information entropy detection module, and use the frame level entropy value to determine whether the speech data is fake speech. The frame level entropy value is a Gaussian distribution.

[0041] Furthermore, the decision module includes:

[0042] The input unit is used to input the target speech frame sequence into the adaptive pre-trained speech detection model to obtain the corresponding embedding vector.

[0043] The calculation unit inputs the embedding vector into the frame horizontal information entropy detection module and calculates the frame horizontal entropy value corresponding to the embedding vector;

[0044] The comparison unit is used to compare the frame horizontal entropy value with a preset threshold.

[0045] The first decision unit is used to determine that the first frame's horizontal entropy value is real speech if the value is greater than the preset threshold.

[0046] The second decision unit is used to determine that the speech is fake if the horizontal entropy value of the first frame is less than or equal to the preset threshold.

[0047] To address the aforementioned technical problems, this application also provides a computer device, including a memory and a processor. The memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the human voice detection method as described in the first aspect.

[0048] To address the aforementioned technical problems, embodiments of this application also provide a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the human voice detection method as described in the first aspect.

[0049] Compared with the prior art, the embodiments of this application have the following main technical effects:

[0050] An adaptive pre-trained speech detection model is constructed by setting an adaptive module in the transformer module of the pre-trained speech model to extract speech feature information. The human voice detection model of this application is constructed by using the frame-level information entropy detection module, which improves the generalization of human voice recognition and detection. By using the joint loss function to train the human voice detection model, the target human voice detection model is obtained, which makes the target human voice detection model have stronger robust performance. Attached Figure Description

[0051] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of an embodiment of a human voice detection method according to this application;

[0053] Figure 2 yes Figure 1 A flowchart of a specific implementation of S100;

[0054] Figure 3 yes Figure 1 A flowchart of a specific implementation of S200;

[0055] Figure 4 yes Figure 1 A flowchart of a specific implementation of the S400;

[0056] Figure 5 yes Figure 1 A flowchart of a specific implementation of the S500;

[0057] Figure 6 yes Figure 5 A flowchart of a specific implementation of S520;

[0058] Figure 7 This is a schematic diagram of the structure of the adaptive pre-trained speech detection model of this application;

[0059] Figure 8 This is a schematic diagram of an embodiment of the first adaptive module in the adaptive pre-trained speech detection model of this application;

[0060] Figure 9 This is a schematic diagram of an embodiment of the second adaptive module in the adaptive pre-trained speech detection model of this application;

[0061] Figure 10 This is a schematic diagram of the frame-level information entropy detection module of this application;

[0062] Figure 11 A schematic diagram of a human voice detection device according to this application;

[0063] Figure 12 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0064] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are all within the scope of protection of this invention.

[0065] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings of this application are intended to cover non-exclusive inclusion.

[0067] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0068] The adaptive parameter fine-tuning method provided in this application can be applied to computer devices with instruction or program execution capabilities. These computer devices can be, but are not limited to, various servers, workstations, personal computers, laptops, etc. The difference in execution between different computing devices is merely a difference in the execution entity; those skilled in the art can foresee that running the solution on different computing devices will produce the same technical effect. The invention will now be described in detail through specific embodiments.

[0069] The purpose of this application is to provide a method, apparatus, device, and medium for human voice detection to solve the problems of cumbersome implementation and poor detection and recognition effect of current deepfake human voice detection technology.

[0070] To address the aforementioned technical problems, this application provides a method for human voice detection, employing the following technical solution:

[0071] In this embodiment of the application, the input speech is detected and recognized using the constructed human voice detection model, which generally includes four stages: data preparation, model construction, model training, and scoring and evaluation.

[0072] In this embodiment of the application, the computer device on which the voice detection method runs (e.g., Figure 12 The network module of the computer device shown can be connected via wired or wireless means. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future wireless connection methods.

[0073] like Figure 1 , Figure 1 This is a flowchart of an embodiment of a voice detection method according to this application. The voice detection method includes the following steps:

[0074] S100. Obtain the speech dataset and process it to obtain the target speech frame sequence. The speech dataset includes the fake speech and the corresponding real speech.

[0075] In some alternative implementations, such as Figure 2 , Figure 2 yes Figure 1 A flowchart of a specific implementation of S100; S100 includes: S110, sampling the speech of the speech dataset according to a preset sampling rate to obtain a speech frame sequence; S120, normalizing the speech frame sequence to obtain a target speech frame sequence of a specified length.

[0076] In this embodiment, acquiring the speech dataset requires collecting a large amount of both fake and real speech. The main sources of fake speech can include text-to-speech (TTS) and voice conversion (VC). When acquiring synthetic speech, it can be obtained by crawling, filtering, and cleaning fake speech already emerging on the internet. When acquiring voice conversion data, advanced speech generation or conversion models, such as ChatTTS and SoVITS, can be used. Fake speech can be created using existing high-quality speech datasets. Alternatively, open-source deepfake speech datasets, such as the DECRO1 dataset, can be used. The acquired speech data is resampled uniformly at a sampling rate of 16kHz, and then normalized by truncation or zero-padding to a fixed length.

[0077] In this embodiment, normalizing the speech frame sequence makes the data distribution more concentrated within a specific range, allowing the voice detection model to more effectively adjust weights and biases during training and reducing the number of iterations required to reach the optimal solution. It also reduces the impact of outliers and noise in the data on the model.

[0078] S200. Set a first adaptive module and a second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model.

[0079] The structure of pre-trained speech models can vary depending on the research institution and task requirements. Different models may employ different neural network architectures, pre-training objectives, and fine-tuning strategies to achieve better performance and generalization capabilities.

[0080] Currently, the vast majority of pre-trained large speech models are based on the Transformer architecture and trained through self-supervised methods. The Transformer model is a widely used neural network architecture in deep learning, particularly adept at handling Natural Language Processing (NLP) tasks such as machine translation, text generation, and understanding. The Transformer model captures dependencies in sequential data using a technique called "attention mechanism." Unlike traditional Recurrent Neural Networks (RNNs), it does not need to process data sequentially but can process the entire sequence simultaneously, significantly improving processing efficiency. In this embodiment, a multi-head attention layer is introduced into the Transformer model, an extension of the self-attention mechanism. It divides the input sequence into multiple heads, each performing self-attention computation independently, then concatenates the results and performs a linear transformation to obtain the final output. This allows the model to capture information from the sequence from multiple perspectives, enhancing its representational capabilities.

[0081] like Figure 7 , Figure 7 This is a schematic diagram of the adaptive pre-trained speech detection model of this application. The pre-trained speech detection model adopts a convolutional neural network architecture, which includes, in sequence:

[0082] Convolutional layer, activation function layer, third normalization layer, first fully connected layer, multi-head attention layer, first adaptive module, first normalization layer, first fully connected layer, second adaptive module, second normalization layer.

[0083] Specifically, in some alternative implementations, such as Figure 3 , Figure 3 yes Figure 1 A flowchart of a specific implementation of S200; S200 includes:

[0084] S210. Connect the output of the second fully connected layer to the input of the first residual layer, and connect the output of the first residual layer to the input of the third fully connected layer to obtain the first adaptive module; S220. Connect the first adaptive module between the multi-head attention layer and the first normalization layer; S230. Connect the output of the fourth fully connected layer to the input of the second residual layer, and connect the output of the second residual layer to the input of the fifth fully connected layer to obtain the second adaptive module; S240. Connect the second adaptive module between the first fully connected layer and the second normalization layer to generate an adaptive pre-trained speech detection model.

[0085] By adding two fully connected layers, the model's structure and parameters can be adjusted according to the specific task requirements, enabling the voice detection model to better adapt to different task needs. For example, for speech recognition tasks, a fully connected layer can be added to learn character or word-level features; for sentiment analysis tasks, a fully connected layer can be added to capture emotional features in speech.

[0086] In this embodiment, the first audio feature includes the input speech data features and the supervised learning features extracted by the multi-head attention layer.

[0087] In this embodiment, as Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the first adaptive module in the adaptive pre-trained speech detection model of this application. The structure of the first adaptive module is: a second fully connected layer - a first residual layer - a third fully connected layer. The second fully connected layer maps the input speech data features and the supervision features extracted by the multi-head attention layer from the high-dimensional space to a new feature space. By learning the weight matrix and bias terms, it performs a linear transformation on the input data to extract preliminary speech input features and supervision features. In addition, the second fully connected layer in the first adaptive module performs feature compression and dimensionality reduction on the above-mentioned speech input features and supervision features, reducing the burden of subsequent computation and improving the efficiency and generalization ability of the model. The first residual layer adds the input features to the features processed by the second fully connected layer, so that the features of the lower layer can be directly passed to the higher layer, avoiding information loss.

[0088] The third fully connected layer further processes the output of the residual layer, mapping the compressed speech input features and supervision features back to the original dimension, and extracting higher-level feature representations.

[0089] In this embodiment, by setting a first adaptive module between the multi-head attention layer and the first normalization layer of the converter module, the voice detection model of this application can extract better features for downstream tasks. Setting the first adaptive module also enhances the supervised learning effect of the voice detection model, enabling more accurate extraction of audio features from the input speech.

[0090] In this embodiment, as Figure 9 , Figure 9 This is a schematic diagram of an embodiment of the second adaptive module in the adaptive pre-trained speech detection model of this application; the structure of the second adaptive module is: fourth fully connected layer - second residual layer - fifth fully connected layer. After passing through the second adaptive module, the output of the adaptive pre-trained speech detection model includes the sixth fully connected layer, the fourth fully connected layer, the second residual layer and the fifth fully connected layer in sequence.

[0091] In this embodiment, by connecting the second adaptive module between the sixth fully connected layer and the second normalization layer, the quality and richness of feature fusion are improved.

[0092] S300. Set a frame-level information entropy detection module at the output of the adaptive pre-trained speech detection model to obtain the human voice detection model.

[0093] Specifically, the frame-level information entropy detection module is connected to the output of the second normalization layer of the adaptive pre-trained speech detection model to form the structure of the aforementioned human voice detection model.

[0094] S400. Obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model.

[0095] In this embodiment, the Adam optimization module (Adaptive Moment Estimation) considers the trend of previous parameter changes when updating parameters. By maintaining an exponentially weighted moving average momentum variable, parameter updates are more stable in the gradient direction, avoiding oscillations near local optima. The learning rate is automatically adjusted according to different parameters. It maintains an exponentially weighted moving average gradient squared variable for each parameter to estimate the parameter's second moment. By combining the estimates of the first moment (momentum) and the second moment, Adam can calculate an adaptive learning rate for each parameter, allowing the learning rate to be dynamically adjusted based on changes in the gradient of different parameters.

[0096] When using the Adam optimization module, it's typically necessary to set some hyperparameters, such as the learning rate, and the decay coefficients of the first and second moments. Generally, you can start by training with the default hyperparameter settings and then adjust them based on the model's performance and convergence.

[0097] After training the voice detection model in this application embodiment, an accuracy or error rate threshold can be set, and the model can be scored and evaluated.

[0098] In some preferred embodiments, as shown in the figure, Figure 4 yes Figure 1 A flowchart of a specific implementation of the S400;

[0099] S400 includes: S410, calculating the cross-entropy between the real speech category and the predicted speech category using the predicted probability distribution of the target human voice detection model, and obtaining the classification loss function; S420, calculating the first MEL spectrum of the predicted speech and the second MEL spectrum of the real speech, calculating the mean square error of the first MEL spectrum and the second MEL spectrum, and obtaining the reconstruction loss function; S430, obtaining the mean and variance of the Gaussian distribution of the frame level entropy values, and calculating the KL loss function based on the mean and variance; S440, adding the classification loss function, the reconstruction loss function, and the KL loss function to obtain the joint loss function.

[0100] In this embodiment, the classification loss function is shown in equation (1):

[0101]

[0102] Wherein, p i Let w represent the probability of predicting real speech, and let cross-entropy be used as the classification loss function. Let N represent the number of categories, w represent the weight of the i-th category, and y represent the index of the real category.

[0103] In this embodiment, the mean square error between the Mel spectra of the forged speech is used as the reconstruction loss function, as shown in equation (2):

[0104]

[0105] Among them, mel gt For real speech mel spectra, mel pred || represents the MEL spectrum (prediction) output by the model.

[0106] In this embodiment, the Gaussian distribution KL loss function is shown in equation (3):

[0107]

[0108] Wherein, σ is the variance of the Gaussian distribution of the horizontal entropy value of the frame, and μ is its mean.

[0109] By adding the three loss functions mentioned above, the final joint loss function is obtained.

[0110] S500. Input the target speech frame sequence into the target human voice detection model, use the frame level information entropy detection module to obtain the frame level entropy value, and use the frame level entropy value to determine whether the speech data is fake speech. The frame level entropy value is Gaussian distributed.

[0111] An adaptive pre-trained speech detection model is constructed by setting an adaptive module in the transformer module of the pre-trained speech model to extract speech feature information. The human voice detection model of this application is constructed by using the frame-level information entropy detection module, which improves the generalization of human voice recognition and detection. By using the joint loss function to train the human voice detection model, the target human voice detection model is obtained, which makes the target human voice detection model have stronger robust performance.

[0112] In some alternative implementations, such as Figure 5 , Figure 5 yes Figure 1 A flowchart of a specific implementation of S500 is provided; S500 includes: S510, inputting the target speech frame sequence into an adaptive pre-trained speech detection model to obtain the corresponding embedding vector; S520, inputting the embedding vector into a frame horizontal information entropy detection module to obtain the frame horizontal entropy value of the corresponding embedding vector; S530, comparing the frame horizontal entropy value with a preset threshold; S540, if the frame horizontal entropy value is greater than the preset threshold, it is determined to be real speech; S550, if the frame horizontal entropy value is less than or equal to the preset threshold, it is determined to be fake speech.

[0113] In some alternative implementations, such as Figure 6 , Figure 6 yes Figure 1 A flowchart of a specific implementation of S520; S520 includes: S521, inputting the embedding vector into the mean encoder and the variance encoder respectively to obtain the corresponding frame horizontal Gaussian distribution; S522, obtaining the first frame horizontal entropy value; wherein, the covariance matrix of the frame horizontal Gaussian distribution is a diagonal matrix, such as... Figure 11 ,like Figure 11 , Figure 11 This is a schematic diagram of the frame-level information entropy detection module of this application.

[0114] In this embodiment, the frame-level information entropy detection module is used to predict whether the input speech data is forged. In reality, forged data and real data differ significantly in the amount of information they contain. This is because generated speech is designed to produce sounds that more closely resemble human hearing, while real speech is a record of the real world and therefore contains more sonic features.

[0115] Specifically, the obtained embedding vectors are input into the mean encoder and variance encoder respectively to obtain a frame-level high-dimensional Gaussian distribution. Then, the entropy corresponding to this Gaussian distribution is obtained using equation (4):

[0116]

[0117] in, This represents the obtained high-dimensional Gaussian distribution, where k represents the dimension number and σ represents the number of dimensions.i Let represent the variance of the i-th dimension.

[0118] In this embodiment, the information entropy of the embedding vector of the input speech is obtained through the frame-level information entropy detection module to perform speech forgery detection, thereby improving the generalization of the human voice detection model.

[0119] In this embodiment, real speech contains more information than fake or synthesized speech in practice. The judgment is made by comparing the frame level entropy value with a preset threshold, thereby improving the generalization of the human voice detection model.

[0120] To solve the above technical problems, such as Figure 11 , Figure 11 A schematic diagram of the structure of a voice detection device 600 based on frame-level information entropy according to this application; embodiments of this application also provide a voice detection device 600 based on frame-level information entropy, employing the voice detection method of the first aspect, including:

[0121] The first acquisition module 601 is used to acquire a speech dataset and process the speech dataset to acquire a target speech frame sequence. The speech dataset includes fake speech and the corresponding real speech.

[0122] The second acquisition module 602 is used to set a first adaptive module and a second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model.

[0123] The third acquisition module 603 is used to set the frame-level information entropy detection module at the output end of the adaptive pre-trained speech detection model to obtain the human voice detection model.

[0124] Training module 604 is used to obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model;

[0125] The decision module 605 is used to input the target speech frame sequence into the target human voice detection model, obtain the frame level entropy value using the frame level information entropy detection module, and use the frame level entropy value to determine whether the speech data is fake speech. The frame level entropy value is Gaussian distributed.

[0126] In some preferred embodiments, the decision module includes:

[0127] The input unit is used to input the target speech frame sequence into the adaptive pre-trained speech detection model to obtain the corresponding embedding vector; the calculation unit inputs the embedding vector into the frame horizontal information entropy detection module to calculate the frame horizontal entropy value of the corresponding embedding vector; the comparison and decision unit is used to compare the frame horizontal entropy value with a preset threshold. If the frame horizontal entropy value is greater than the preset threshold, it is judged as real speech; if the frame horizontal entropy value is less than or equal to the preset threshold, it is judged as fake speech.

[0128] To address the aforementioned issues, this application also proposes a computer device that employs the following technical solution: it includes a processor, a network module, and a memory, with the processor and memory interconnected via the network module.

[0129] This computer device can be a computer, server, workstation, or other similar device; it can also be a mobile device such as a mobile phone, tablet, or in-vehicle mobile terminal; or other devices with program execution capabilities. Its internal structure diagram can be as follows: Figure 12 As shown, Figure 12 This is a schematic diagram of a computer device according to an embodiment of the present application. The computer device includes a processor, a memory, and a network module. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, instructions, or code. The internal memory provides an environment for the operation of the operating system and instructions or code in the non-volatile storage media. When the instructions or code are executed by the processor, they implement the functions or steps of the aforementioned voice detection method. The network module of the computer device may include a network interface and / or a wireless network module, allowing the computer device to communicate with other devices or service platforms. Furthermore, the computer device may also include a display screen and input devices, etc.

[0130] The memory is used to store computer programs, which include program instructions. The processor is configured to call the program instructions, and when the processor executes the instructions or code, it implements the steps of the human voice detection method described above.

[0131] To address the aforementioned problems, this application also proposes a computer-readable storage medium, employing the following technical solution: the computer-readable storage medium stores a computer program, the computer program including program instructions, which are implemented when executed by a processor. Figures 1 to 6 The voice detection methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0132] The aforementioned computer-readable storage medium can be a voice detection device based on frame-level information entropy provided in any of the foregoing embodiments, or an internal storage unit of the aforementioned terminal device, such as a hard drive or memory of a computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device.

[0133] Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0134] However, it should be understood that it is not required to implement all the components shown; more or fewer components may be implemented instead. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0135] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control. Compared with the prior art, the embodiments of this application have the following main technical effects:

[0136] An adaptive pre-trained speech detection model is constructed by setting an adaptive module in the transformer module of the pre-trained speech model to extract speech feature information. The human voice detection model of this application is constructed by using the frame-level information entropy detection module, which improves the generalization of human voice recognition and detection. By using the joint loss function to train the human voice detection model, the target human voice detection model is obtained, which makes the target human voice detection model have stronger robust performance.

[0137] The software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0138] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for human voice detection, characterized in that it includes: Acquire a voice dataset and process the voice dataset, which includes fake voice and corresponding real voice; A first adaptive module and a second adaptive module are set between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model; A frame-level information entropy detection module is set at the output of the adaptive pre-trained speech detection model to obtain a human voice detection model. Obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model; The target speech frame sequence of the speech data to be detected is input into the target human voice detection model. The first frame horizontal entropy value of the speech data to be detected is obtained by the frame horizontal information entropy detection module. The first frame horizontal entropy value is used to determine whether the speech data to be detected is fake speech. The first frame horizontal entropy value is Gaussian distributed.

2. The human voice detection method according to claim 1, characterized in that, The step of processing the speech dataset to obtain the target speech frame sequence includes: The speech in the speech dataset is sampled according to a preset sampling rate to obtain the speech frame sequence; The speech frame sequence is normalized to obtain a target speech frame sequence of a specified length.

3. The human voice detection method according to claim 1, characterized in that, The method involves setting a first adaptive module and a second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model, including: Connect the output of the second fully connected layer to the input of the first residual layer, and connect the output of the first residual layer to the input of the third fully connected layer to obtain the first adaptive module; Connect the first adaptive module between the multi-head attention layer and the first normalization layer; Connect the output of the fourth fully connected layer to the input of the second residual layer, and connect the output of the second residual layer to the input of the fifth fully connected layer to obtain the second adaptive module; The second adaptive module is connected between the first fully connected layer and the second normalized layer to generate the adaptive pre-trained speech detection model.

4. The human voice detection method according to claim 1, characterized in that, The process of obtaining the joint loss function includes: The cross-entropy between the real speech category and the predicted speech category is calculated using the predicted probability distribution to obtain the classification loss function; Calculate the first MEL spectrum of the predicted speech and the second MEL spectrum of the real speech, calculate the mean square error of the first MEL spectrum and the second MEL spectrum, and obtain the reconstruction loss function. Obtain the mean and variance of the Gaussian distribution of the horizontal entropy value of the second frame, and calculate the KL loss function based on the mean and variance; The classification loss function, the reconstruction loss function, and the KL loss function are added together to obtain the joint loss function; The second frame horizontal entropy value is the frame horizontal entropy value of the speech data embedding vector during the training phase.

5. The human voice detection method according to claim 1, characterized in that, The step involves inputting the target speech frame sequence of the speech data to be detected into the target human voice detection model, using the frame-level information entropy detection module to obtain the first frame-level entropy value of the speech data to be detected, and using the first frame-level entropy value to determine whether the speech data to be detected is fake speech. The first frame-level entropy value is a Gaussian distribution, including: The target speech frame sequence is input into the adaptive pre-trained speech detection model to obtain the corresponding embedding vector; The embedding vector is input into the frame horizontal information entropy detection module to calculate the first frame horizontal entropy value corresponding to the embedding vector. Compare the horizontal entropy value of the first frame with a preset threshold; If the horizontal entropy value of the first frame is greater than the preset threshold, it is determined to be real speech; If the horizontal entropy value of the first frame is less than or equal to the preset threshold, it is determined to be fake speech.

6. The human voice detection method according to claim 5, characterized in that, The step of inputting the embedding vector into the frame horizontal information entropy detection module and calculating the first frame horizontal entropy value corresponding to the embedding vector includes: The embedding vectors are input into the mean encoder and the variance encoder respectively to obtain the corresponding frame level Gaussian distribution. Calculate the entropy of the horizontal Gaussian distribution of the frame to obtain the horizontal entropy value of the first frame; The covariance matrix of the Gaussian distribution of the horizontal entropy values ​​in the first frame is a diagonal matrix.

7. A human voice detection device, employing the human voice detection method according to any one of claims 1-6, characterized in that, include: The first acquisition module is used to acquire a voice dataset and process the voice dataset, wherein the voice dataset includes fake voice and corresponding real voice; The second acquisition module is used to set the first adaptive module and the second adaptive module between the multi-head attention layer and the first normalization layer, and between the first fully connected layer and the second normalization layer of the transformer model, respectively, to generate an adaptive pre-trained speech detection model. The third acquisition module is used to set a frame-level information entropy detection module at the output end of the adaptive pre-trained speech detection model to obtain a human voice detection model. The training module is used to obtain the joint loss function, train the human voice detection model using the speech dataset and the joint loss function, and optimize it using the Adam optimization module to obtain the target human voice detection model. The decision module is used to input the target speech frame sequence of the speech data to be detected into the target human voice detection model, use the frame level information entropy detection module to obtain the first frame level entropy value of the speech data to be detected, and use the first frame level entropy value to determine whether the speech data to be detected is fake speech. The first frame level entropy value is Gaussian distributed.

8. The human voice detection device according to claim 7, characterized in that, The judgment module includes: The input unit is used to input the target speech frame sequence into the adaptive pre-trained speech detection model to obtain the corresponding embedding vector. The calculation unit inputs the embedding vector into the frame horizontal information entropy detection module and calculates the first frame horizontal entropy value corresponding to the embedding vector. The comparison unit is used to compare the horizontal entropy value of the first frame with a preset threshold. The first decision unit is used to determine that the first frame's horizontal entropy value is real speech if the value is greater than the preset threshold. The second decision unit is used to determine that the speech is fake if the horizontal entropy value of the first frame is less than or equal to the preset threshold.

9. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the human voice detection method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the human voice detection method as described in any one of claims 1 to 6.

Images