Multiple adversarial discrimination forged audio detection system

Through the multiple adversarial discriminant forged audio detection system, the data acquisition, waveform detection and time domain feature modeling modules combined with adversarial neural networks are used to solve the problem that traditional methods cannot detect deep forged audio, achieving higher detection accuracy and robustness sex.

CN118280389BActive Publication Date: 2024-11-15NANJING LONGYUAN INFORMATION TECH CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202410365711.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-28
Publication Date
2024-11-15
Estimated Expiration
2044-03-28

AI Technical Summary

Technical Problem

Traditional forged audio detection methods cannot effectively detect deep forged audio, resulting in insufficient detection accuracy and robustness.

Method used

Multiple adversarial discriminant forged audio detection system, including data acquisition module, waveform detection model, time domain feature modeling module and adversarial neural network classifier, use deep neural networks by preprocessing, spectrum analysis and feature modeling of audio data, using deep neural networks Confrontation and discrimination forged audio.

Benefits of technology

Improves the detection accuracy and robustness of deep audio forgery, and can effectively identify real and forged audio.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118280389B_ABST
    Figure CN118280389B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of speech recognition technology, and in particular to a multiple adversarial discrimination forged audio detection system; the system comprises a data acquisition module, a waveform detection model, a time domain feature modeling module and an adversarial neural network classifier; the waveform detection model is used to extract the spectrum in the preprocessed original audio data, and judge whether the original audio is real audio based on the continuity of the spectrum; the time domain feature modeling module is used to model the audio waveform and features; the adversarial neural network classifier is used to judge the features, and again judge whether the original audio is real audio; by performing waveform spectrum analysis on the target audio, adopting a deep neural network to model the audio waveform and features, and using the adversarial neural network to judge the features, it is judged whether the audio source is forged, and the forgery of the deep audio is detected, thereby improving the accuracy and robustness of the detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular to a multiple adversarial discrimination forged audio detection system. Background Art

[0002] With the rapid development of deepfake technology, voice forgery has become one of the huge risks faced by all walks of life. Traditional forged audio detection methods are mainly based on hand-designed feature extraction and statistical models, which are less effective against advanced deepfake technologies. Standard hand-crafted features such as Linear Frequency Cepstral Coefficients (LFCC) and Constant Q Cepstral Coefficients (CQCC) can capture frequency domain features, but smooth the speech signal and hinder the extraction of narrowband speaker features such as pitch and formants.

[0003] In summary, traditional forged audio detection methods are largely unable to detect deep audio forgeries. Therefore, a multi-adversarial discriminative forged audio detection system is needed to improve the accuracy and robustness of detection. Summary of the invention

[0004] The purpose of the present invention is to provide a multiple adversarial discrimination forged audio detection system, which solves the problem that traditional forged audio detection methods are largely unable to detect deep audio forgery.

[0005] To achieve the above-mentioned purpose, the present invention adopts a multiple adversarial discrimination forged audio detection system, including a data acquisition module, a waveform detection model, a time domain feature modeling module and an adversarial neural network classifier, wherein the waveform detection model is connected to the data acquisition module, the time domain feature modeling module is connected to the waveform detection model, and the adversarial neural network classifier is connected to the time domain feature modeling module;

[0006] The data acquisition module is used to collect original audio data, pre-process the original audio data, and output an audio signal;

[0007] The waveform detection model is used to extract the frequency spectrum in the preprocessed original audio data, and determine whether the original audio is real audio based on the continuity of the frequency spectrum;

[0008] The time domain feature modeling module is used to model the audio waveform and features;

[0009] The adversarial neural network classifier is used to distinguish the features and again determine whether the original audio is real audio.

[0010] The data acquisition module includes an original audio acquisition unit and a preprocessing unit, and the preprocessing unit is connected to the original audio acquisition unit and the waveform detection model respectively.

[0011] Wherein, the original audio acquisition unit is used to collect original audio data;

[0012] The preprocessing unit is used to perform audio gain, noise suppression, and mixing operations on the original audio data to improve the quality of the audio data and output an audio signal.

[0013] Among them, the waveform detection model includes a decomposition unit, a spectrum information acquisition unit, a spectrum feature extraction unit, a time domain information detection unit and a discrimination unit. The decomposition unit is connected to the data acquisition module, the spectrum information acquisition unit and the time domain information detection unit are respectively connected to the decomposition unit, the spectrum feature extraction unit is connected to the spectrum information acquisition unit, and the spectrum feature extraction unit and the time domain information detection unit are respectively connected to the discrimination unit.

[0014] Wherein, the decomposition unit is used to decompose the audio signal into short time frames;

[0015] The spectrum information acquisition unit is used to apply Fourier transform to each frame to acquire spectrum information;

[0016] The spectrum feature extraction unit is used to calculate the energy value of each frame and convert it into a logarithmic form to enhance the stability and robustness of the feature, and respectively extract the power spectrum feature, the amplitude spectrum feature and the phase spectrum feature;

[0017] The time domain information detection unit is used to detect the connectivity of the audio signal in time, including jitter and intermittency;

[0018] The determination unit is used to determine whether the original audio is real audio according to power spectrum features, amplitude spectrum features, phase spectrum features and connectivity.

[0019] Among them, the time domain feature modeling module includes multiple groups of convolution units, pooling units and time-frequency global information modeling units, and the multiple groups of convolution units are connected in sequence. The convolution units at the beginning are connected to the waveform detection model, and the convolution units at the end are connected to the pooling units. The time-frequency global information modeling unit is respectively connected to the pooling unit and the adversarial neural network classifier.

[0020] Wherein, the plurality of groups of convolution units are used to extract local features of the audio using a group of filters, including frequency, intonation, and resonance;

[0021] The pooling unit is used to further extract significant features to improve the accuracy and efficiency of detection;

[0022] The time-frequency global information modeling unit is used to randomly discard neurons after the maximum pooling layer to reduce the overfitting risk of the model.

[0023] The present invention discloses a multiple adversarial discrimination forged audio detection system, wherein the data acquisition module is used to acquire original audio data, preprocess the original audio data, and output an audio signal; the waveform detection model is used to extract the spectrum in the preprocessed original audio data, and judge whether the original audio is real audio based on the continuity of the spectrum; the time domain feature modeling module is used to model the audio waveform and features; the adversarial neural network classifier is used to judge the features, and again judge whether the original audio is real audio; by performing waveform spectrum analysis on the target audio, using a deep neural network to model the audio waveform and features, and using an adversarial neural network to judge the features, it is possible to judge whether the audio source is forged, thereby detecting the forgery of the deep audio, and improving the accuracy and robustness of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a structural schematic diagram of the multiple adversarial discrimination counterfeit audio detection system of the present invention.

[0026] 100-data acquisition module, 101-original audio acquisition unit, 102-preprocessing unit, 200-waveform detection model, 201-decomposition unit, 202-spectral information acquisition unit, 203-spectral feature extraction unit, 204-time domain information detection unit, 205-discrimination unit, 300-time domain feature modeling module, 301-convolution unit, 302-pooling unit, 303-time-frequency global information modeling unit, 400-adversarial neural network classifier. DETAILED DESCRIPTION

[0027] See also Figure 1 ,in Figure 1 It is a structural schematic diagram of the multiple adversarial discrimination counterfeit audio detection system of the present invention.

[0028] The present invention provides a multi-adversarial discrimination forged audio detection system, comprising a data acquisition module 100, a waveform detection model 200, a time domain feature modeling module 300 and an adversarial neural network classifier 400, wherein the waveform detection model 200 is connected to the data acquisition module 100, the time domain feature modeling module 300 is connected to the waveform detection model 200, and the adversarial neural network classifier 400 is connected to the time domain feature modeling module 300;

[0029] The data acquisition module 100 is used to collect original audio data, pre-process the original audio data, and output an audio signal;

[0030] The waveform detection model 200 is used to extract the spectrum in the preprocessed original audio data, and determine whether the original audio is real audio based on the continuity of the spectrum;

[0031] The time domain feature modeling module 300 is used to model the audio waveform and features;

[0032] The adversarial neural network classifier 400 is used to identify features and again determine whether the original audio is real audio.

[0033] In this embodiment, the data acquisition module 100 acquires raw audio data, preprocesses the raw audio data, and outputs an audio signal; the waveform detection model 200 extracts the spectrum in the preprocessed raw audio data, and determines whether the raw audio is real audio based on the continuity of the spectrum; the time domain feature modeling module 300 models the audio waveform and features; the adversarial neural network classifier 400 distinguishes the features and again determines whether the original audio is real audio; by performing waveform spectrum analysis on the target audio, using a deep neural network to model the audio waveform and features, and using an adversarial neural network to distinguish the features, it is possible to determine whether the audio source is forged, thereby detecting the forgery of the deep audio, and improving the accuracy and robustness of the detection.

[0034] Furthermore, the data acquisition module 100 includes an original audio acquisition unit 101 and a preprocessing unit 102 , and the preprocessing unit 102 is connected to the original audio acquisition unit 101 and the waveform detection model 200 , respectively.

[0035] Furthermore, the original audio acquisition unit 101 is used to collect original audio data;

[0036] The pre-processing unit 102 is used to perform audio gain, noise suppression, and mixing operations on the original audio data to improve the quality of the audio data and output an audio signal.

[0037] In this embodiment, the original audio acquisition unit 101 collects original audio data; the pre-processing unit 102 performs audio gain, noise suppression, and mixing operations on the original audio data to improve the quality of the audio data and output an audio signal.

[0038] Furthermore, the waveform detection model 200 includes a decomposition unit 201, a spectrum information acquisition unit 202, a spectrum feature extraction unit 203, a time domain information detection unit 204 and a discrimination unit 205, the decomposition unit 201 is connected to the data acquisition module 100, the spectrum information acquisition unit 202 and the time domain information detection unit 204 are respectively connected to the decomposition unit 201, the spectrum feature extraction unit 203 is connected to the spectrum information acquisition unit 202, and the spectrum feature extraction unit 203 and the time domain information detection unit 204 are respectively connected to the discrimination unit 205.

[0039] Further, the decomposition unit 201 is used to decompose the audio signal into short time frames;

[0040] The spectrum information acquisition unit 202 is used to apply Fourier transform to each frame to acquire spectrum information;

[0041] The spectrum feature extraction unit 203 is used to calculate the energy value of each frame and convert it into a logarithmic form to enhance the stability and robustness of the feature, and respectively extract the power spectrum feature, the amplitude spectrum feature and the phase spectrum feature;

[0042] The time domain information detection unit 204 is used to detect the connectivity of the audio signal in time, including jitter and intermittency;

[0043] The determination unit 205 is used to determine whether the original audio is real audio according to power spectrum features, amplitude spectrum features, phase spectrum features and connectivity.

[0044] In this embodiment, the decomposition unit 201 decomposes the audio signal into short time frames; the spectrum information acquisition unit 202 applies Fourier transform to each frame to obtain spectrum information; the spectrum feature extraction unit 203 calculates the energy value of each frame and converts it into logarithmic form to enhance the stability and robustness of the feature, and extracts power spectrum features, amplitude spectrum features and phase spectrum features respectively; wherein the power spectrum feature represents the energy distribution of the sound signal, the amplitude spectrum feature represents the amplitude distribution of the sound signal, and the phase spectrum feature represents the phase distribution of the sound signal in time and frequency; the time domain information detection unit 204 detects the connectivity of the audio signal in time, including jitter and intermittent, and the above features can reflect the energy distribution, amplitude distribution, phase distribution and time continuity changes of the sound signal, thereby realizing the detection and identification of forged speech; the discrimination unit 205 discriminates whether the original audio is real audio according to the power spectrum feature, amplitude spectrum feature, phase spectrum feature and connectivity.

[0045] Furthermore, the time domain feature modeling module 300 includes multiple groups of convolution units 301, pooling units 302 and time-frequency global information modeling units 303, and the multiple groups of convolution units 301 are connected in sequence. The convolution units 301 at the beginning are connected to the waveform detection model 200, and the convolution units 301 at the end are connected to the pooling units 302. The time-frequency global information modeling unit 303 is respectively connected to the pooling unit 302 and the adversarial neural network classifier 400.

[0046] Further, the plurality of groups of convolution units 301 are used to extract local features of the audio using a group of filters, including frequency, intonation, and resonance;

[0047] The pooling unit 302 is used to further extract significant features to improve the accuracy and efficiency of detection;

[0048] The time-frequency global information modeling unit 303 is used to randomly discard neurons after the maximum pooling layer to reduce the overfitting risk of the model.

[0049] In this embodiment, multiple groups of the convolution units 301 use a group of filters to extract local features of the audio, including frequency, intonation, and resonance. Multiple groups of the convolution units 301 use multiple filters. The convolution operation is based on the idea of ​​a sliding window. The filter is multiplied element-by-element with different positions of the input audio and summed to generate an output feature map. Filters of different sizes can capture features on different time scales. For example, smaller filters can capture short-term rapid changes, while larger filters can capture long-term trends and patterns. By using multiple filters in the convolution layer, different types of features can be learned, and each filter can be updated with parameters through back propagation; the convolution layer can also further reduce the dimension of the feature map by using a pooling operation and extract the most significant features; the pooling unit 302 further extracts the most significant features to improve the accuracy and efficiency of detection; the time-frequency global information modeling unit 303 randomly discards neurons after the maximum pooling layer to reduce the risk of overfitting of the model; by extracting local features and learning features on different time scales, real audio and forged audio can be better distinguished. Combined with the pooling operation, the most significant features can be further extracted to improve the accuracy and efficiency of detection.

[0050] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A multi-adversarial discrimination forged audio detection system, It is characterized in that It includes a data acquisition module, a waveform detection model, a time domain feature modeling module and an adversarial neural network classifier, wherein the waveform detection model is connected to the data acquisition module, the time domain feature modeling module is connected to the waveform detection model, and the adversarial neural network classifier is connected to the time domain feature modeling module; The data acquisition module is used to collect original audio data, pre-process the original audio data, and output an audio signal; The waveform detection model is used to extract the frequency spectrum in the preprocessed original audio data, and determine whether the original audio is real audio based on the continuity of the frequency spectrum; The time domain feature modeling module is used to model the audio waveform and features; The adversarial neural network classifier is used to distinguish the features and again determine whether the original audio is real audio; The time domain feature modeling module includes multiple groups of convolution units, pooling units and time-frequency global information modeling units, and the multiple groups of convolution units are connected in sequence, the convolution units at the beginning are connected to the waveform detection model, the convolution units at the end are connected to the pooling units, and the time-frequency global information modeling units are respectively connected to the pooling units and the adversarial neural network classifier; The plurality of groups of convolution units are used to extract local features of the audio using a group of filters, including frequency, intonation, and resonance; The pooling unit is used to further extract significant features to improve the accuracy and efficiency of detection; The time-frequency global information modeling unit is used to randomly discard neurons after the maximum pooling layer to reduce the overfitting risk of the model.

2. The multi-adversarial discrimination forged audio detection system as claimed in claim 1, It is characterized in that The data acquisition module includes an original audio acquisition unit and a preprocessing unit, and the preprocessing unit is connected to the original audio acquisition unit and the waveform detection model respectively.

3. The multi-adversarial discrimination forged audio detection system as claimed in claim 2, It is characterized in that The original audio acquisition unit is used to collect original audio data; The preprocessing unit is used to perform audio gain, noise suppression, and mixing operations on the original audio data to improve the quality of the audio data and output an audio signal.

4. The multi-adversarial discrimination forged audio detection system as claimed in claim 1, It is characterized in that The waveform detection model includes a decomposition unit, a spectrum information acquisition unit, a spectrum feature extraction unit, a time domain information detection unit and a discrimination unit. The decomposition unit is connected to the data acquisition module, the spectrum information acquisition unit and the time domain information detection unit are respectively connected to the decomposition unit, the spectrum feature extraction unit is connected to the spectrum information acquisition unit, and the spectrum feature extraction unit and the time domain information detection unit are respectively connected to the discrimination unit.

5. The multi-adversarial discrimination forged audio detection system as claimed in claim 4, It is characterized in that The decomposition unit is used to decompose the audio signal into short time frames; The spectrum information acquisition unit is used to apply Fourier transform to each frame to acquire spectrum information; The spectrum feature extraction unit is used to calculate the energy value of each frame and convert it into a logarithmic form to enhance the stability and robustness of the feature, and respectively extract the power spectrum feature, the amplitude spectrum feature and the phase spectrum feature; The time domain information detection unit is used to detect the connectivity of the audio signal in time, including jitter and intermittency; The determination unit is used to determine whether the original audio is real audio according to power spectrum features, amplitude spectrum features, phase spectrum features and connectivity.

Citation Information

Patent Citations

  • Method and system for detecting voice spoofing attack of speakers on basis of deep learning

    CN105869630A

  • Artificially synthesized speech detection method and device, computer equipment and storage medium

    CN112185417A

  • Synthetic speech detection method based on deep self-attention neural network classifier

    CN114898773A

  • Speech recognition method and device, model training method and device, medium and electronic equipment

    CN115376498A

  • Synthetic speech detection method and system based on generative adversarial network

    CN115985340A