AI synthetic sound detection method and system based on spectrum disturbance analysis

Abstract

The invention discloses an AI synthetic sound detection method and system based on spectrum disturbance analysis, and relates to the field of audio detection. According to the specific implementation scheme, the method comprises the following steps: S1, collecting an audio signal and preprocessing the audio signal to obtain a standardized audio signal; s2, performing spectrum analysis and spectrum disturbance on the standardized audio signal, and calculating an original spectrogram and a disturbed spectrogram; s3, obtaining a feature vector based on the original spectrogram and the perturbed spectrogram; s4, processing the real audio signal and the AI synthesized audio signal according to the steps S1-S3 to form a feature vector set, and dividing feature vectors in the feature vector set and corresponding labels into a training set and a test set; and S5, training the neural network model by using the training set, verifying and testing the trained neural network model by using the test set, and finally realizing AI voice detection. According to the method, the random disturbance is added, so that the detection model can capture the characteristics of the AI synthetic audio more easily, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of AI audio detection technology, and in particular to a detection method and system for AI synthesized sound based on spectrum disturbance analysis. Background Art

[0002] The explosion of generative AI has greatly promoted the development and application of artificial intelligence technology, especially in the fields of text, images, video and audio, where generative AI can create incredibly realistic content. However, this has also raised a series of security and ethical issues, such as deep fake technology may be used to create false audio, commit fraud, mislead the public or violate personal privacy. Therefore, AI audio detection technology has become particularly important. It can analyze audio files and identify whether generative AI technology is used in them, helping us distinguish the true from the false and prevent potential risks. With the advancement of AI synthetic sound technology, the quality of synthetic sound is getting closer and closer to the real sound, and it is difficult for the human ear to distinguish the slight differences, resulting in a decrease in detection accuracy. Summary of the invention

[0003] Based on this, the present invention provides a method and system for detecting AI synthesized sound based on spectral perturbation analysis to solve the problem of low accuracy in synthesized sound detection.

[0004] In a first aspect, the present invention provides a method for detecting AI synthesized speech based on spectrum disturbance analysis, comprising:

[0005] S1: collecting audio signals and preprocessing the audio signals to obtain standardized audio signals;

[0006] S2: performing spectrum analysis and spectrum perturbation on the standardized audio signal, and calculating an original spectrum graph and a perturbed spectrum graph;

[0007] S3: Obtaining a difference feature vector based on the original spectrum graph and the perturbed spectrum graph;

[0008] S4: Processing the real audio signal and the AI ​​synthesized audio signal according to steps S1 to S3 to form a feature vector set, and dividing the difference feature vectors and corresponding labels in the feature vector set into a training set and a test set;

[0009] S5: Use the training set to train the neural network model, use the test set to verify and test the trained neural network model, and finally realize AI voice detection.

[0010] The specific operations of preprocessing the audio signal include:

[0011] Converting the sampling rate of the audio signal by downsampling, and adjusting the sampling rate of the audio signal to a target sampling rate;

[0012] removing noise from the audio signal after the sampling rate adjustment;

[0013] Amplitude normalization is performed on the audio signal after noise removal.

[0014] The calculation of the original spectrum diagram includes:

[0015] The standardized audio signal is segmented and a window function is applied. The frequency component of each audio signal is obtained by fast Fourier transform, and the power spectrum density is calculated. The power spectrum density at each time point is arranged in chronological order to generate an original spectrum diagram showing the change of signal frequency over time.

[0016] The calculation of the perturbed spectrum diagram includes:

[0017] Adding random noise as a disturbance to the original spectrogram, converting the disturbed spectrogram back into a time-domain audio signal, and finally calculating the disturbed spectrogram;

[0018] Assuming the original spectrum is x(t), the noise amplitude factor is ∈, the calculation of the noise signal n(t) is:

[0019] n(t)=∈·randn(t);

[0020] Where t is any time;

[0021] The noise signal is superimposed on the original spectrum to obtain the disturbed spectrum x'(t).

[0022] x'(t)=x(t)+n(t).

[0023] The acquisition of the difference feature vector includes:

[0024] The frequency domain features and time domain features of the original spectrum graph are extracted, and the spectrum difference features are extracted by comparing the original spectrum graph and the perturbed spectrum graph. The frequency domain features, the time domain features and the spectrum difference features are fused to form a difference feature vector.

[0025] The acquisition of the spectrum difference feature includes:

[0026] The spectrum difference between the original spectrum graph and the spectrum graph after disturbance and the signal-to-noise ratio between the original signal and the signal after disturbance are fused by weighted synthesis to obtain the spectrum difference feature;

[0027] The spectrum difference D spectrum The calculation includes,

[0028] D spectrum =∑f |P original (f)-P perturbed (f)|;

[0029] Among them, P original (f) represents the spectrum of the original spectrogram, P perturbed represents the spectrum of the perturbed spectrum graph, and f represents the frequency;

[0030] The signal-to-noise ratio of SNP dB The calculation includes,

[0031]

[0032] Among them, Power signal Indicates signal power, Power noise Represents the noise power.

[0033] The formation of the difference feature vector specifically includes:

[0034] The time domain feature vector T m , frequency domain feature vector F n and the difference eigenvector D k Perform feature connection to form a comprehensive difference feature vector C,

[0035] C=[T1,T2,...,T m , F1, F2, ..., F n , D1, D2, ..., D k ].

[0036] The neural network model is a neural network model including three fully connected layers, and ReLU is used as an activation function between each fully connected layer.

[0037] A second aspect of the present invention provides an AI synthesized sound detection system based on spectrum disturbance analysis, comprising:

[0038] A signal processing unit, used for collecting audio signals and preprocessing the audio signals to obtain standardized audio signals;

[0039] A difference feature vector acquisition unit, used for performing spectrum analysis and spectrum perturbation on the standardized audio signal, and calculating an original spectrum graph and a perturbed spectrum graph; and acquiring a difference feature vector based on the original spectrum graph and the perturbed spectrum graph;

[0040] The speech detection unit is used to process the real audio signal and the AI ​​synthesized audio signal to form a feature vector set, and divide the difference feature vectors and corresponding labels in the feature vector set into a training set and a test set; use the training set to train the neural network model, and use the test set to verify and test the trained neural network model, and finally realize AI speech detection.

[0041] Beneficial effects: By adding appropriate disturbances, the present invention can make the model learn to ignore these noises during the training process, thereby improving the performance of the model in practical applications; by adding spectral disturbances during the training process, the model can learn to recognize and resist these adversarial attacks, thereby improving the security and reliability of the model; through spectral disturbances, these subtle differences can be amplified, so that the detection model can more easily capture the characteristics of AI synthesized audio, thereby improving the accuracy of detection; by analyzing the contribution of different features in the classification process, the decision-making mechanism of the model can be better understood, and then the model can be improved and optimized. In addition, by fusing multiple features, the impact of the failure of a single feature can be reduced, and the stability and reliability of the model can be improved.

[0042] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0044] Figure 1 is a flow chart provided according to the present invention;

[0045] Figure 2 is a flow chart of preprocessing an audio signal according to the present invention;

[0046] Figure 3 is an original audio graph provided according to the present invention;

[0047] Figure 4 is a spectrum diagram after disturbance provided by the present invention.

[0048] Figure 5 It is a structural diagram of a neural network model provided according to the present invention. DETAILED DESCRIPTION

[0049] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0050] like Figure 1 As shown, in a first aspect of the present invention, a method for detecting AI synthesized sound based on spectrum disturbance analysis is provided, comprising:

[0051] S1: Collect audio signals and pre-process the audio signals to obtain standardized audio signals. It should be noted that:

[0052] The purpose of collecting audio signals and preprocessing them is to ensure the consistency and quality of data and provide clean and standardized input for subsequent analysis. The present invention preprocesses the audio signals by using sampling rate conversion, noise removal and amplitude normalization.

[0053] like Figure 2 As shown, the specific operations of preprocessing the audio signal include:

[0054] Downsampling is used to convert the sampling rate of the audio signal, and the sampling rate of the audio signal is adjusted to a target sampling rate;

[0055] To avoid aliasing, a low-pass filter is applied before downsampling, with a cutoff frequency of half the target sampling rate. The filter can be a FIR filter;

[0056]

[0057] Where y[n] represents the output signal of the discrete time series, n represents the moment of the current sample, h[k] represents the filter coefficient, k represents the index, and the value range is 0 to N-1, x represents the input signal, and x[nk] represents the value of the input signal at time point nk;

[0058] Samples are taken at intervals based on the ratio of the target sampling rate to the original sampling rate.

[0059] y[n]=x[nM];

[0060] Among them, y[n] represents the output signal of the discrete time series, M represents the decimation factor, that is, one sample is taken every M samples, x represents the input signal, and n represents the time of the current sample.

[0061] Removing noise from the audio signal after the sampling rate adjustment;

[0062] Perform amplitude normalization on the audio signal after noise removal.

[0063] Furthermore, the present invention removes background noise in the audio signal by using a filter or signal enhancement technology, mainly using time domain filtering or frequency domain filtering technology to remove background noise;

[0064] Amplitude normalization is to adjust the amplitude of the audio signal to a uniform range, usually between [-1, 1] or [0, 1]. This is useful for preventing overflow in signal processing, reducing the demand for dynamic range, and ensuring the compatibility of the signal on certain devices.

[0065] S2: Perform spectrum analysis and spectrum perturbation on the standardized audio signal, and calculate the original spectrum graph and the perturbed spectrum graph. It should be noted that:

[0066] The calculation of the original spectrogram includes,

[0067] The standardized audio signal is processed in segments and the window function is applied. The frequency components of each audio signal are obtained by fast Fourier transform, and the power spectrum density is calculated. The power spectrum density of each time point is arranged in chronological order to generate the original spectrum diagram showing the change of signal frequency over time (such as Figure 3 shown).

[0068] The calculation of the perturbed spectrum includes,

[0069] The random noise is added as a disturbance to the original spectrogram, and the perturbed spectrogram is converted back to the time domain audio signal. Finally, the perturbed spectrogram (such as Figure 4 shown);

[0070] Furthermore, to enhance the robustness of AI synthetic audio signal detection, spectrum perturbation technology can enhance the robustness of the detection model to different types of noise and interference. In practical applications, audio signals may be affected by various environmental noises. By adding appropriate perturbations, the model can learn to ignore these noises during training, thereby improving the performance of the model in practical applications.

[0071] Assuming the original spectrum is x(t), the noise amplitude factor is ∈, the calculation of the noise signal n(t) is:

[0072] n(t)=∈·randn(t);

[0073] Where t is any time;

[0074] The noise signal is superimposed on the original spectrum to obtain the disturbed spectrum x'(t).

[0075] x'(t)=x(t)+n(t).

[0076] S3: Obtain a difference feature vector based on the original spectrum graph and the perturbed spectrum graph. It should be noted that:

[0077] The acquisition of the difference feature vector includes:

[0078] The frequency domain features and time domain features of the original spectrum graph are extracted. At the same time, the spectrum difference features are extracted by comparing the original spectrum graph and the perturbed spectrum graph. The frequency domain features, time domain features and spectrum difference features are fused to form a difference feature vector.

[0079] Furthermore, by integrating multiple features, the impact of failure of a single feature can be reduced and the stability and reliability of the model can be improved.

[0080] Furthermore, the frequency domain features mainly include frequency center, bandwidth, spectrum peak, spectrum roll-off point, and spectrum flatness. The time domain features reflect the changes of the signal in the time domain, mainly including energy envelope, short-time energy, and zero crossing rate.

[0081] The acquisition of spectrum difference features includes:

[0082] The spectrum difference between the original spectrum graph and the perturbed spectrum graph and the signal-to-noise ratio between the original signal and the perturbed signal are fused by weighted synthesis to obtain the spectrum difference feature;

[0083] Spectrum Difference D spectrum The calculation includes,

[0084] D spectrum =∑ f |P original (f)-P perturbed (f)|;

[0085] Among them, P original (f) represents the spectrum of the original spectrogram, P perturbed represents the spectrum of the perturbed spectrum graph, and f represents the frequency;

[0086] Signal-to-Noise Ratio (SNR) dB The calculation includes,

[0087]

[0088] Among them, Power signal Indicates signal power, Power noise Represents the noise power.

[0089] Furthermore, the operation of fusing the frequency difference and the signal-to-noise ratio by weighted synthesis includes:

[0090] Set the weight of the spectrum difference to ω spectrum , the weight of the signal-to-noise ratio is and Then, the comprehensive difference D total The calculation is:

[0091] The formation of the difference feature vector specifically includes:

[0092] The time domain feature vector T m , frequency domain feature vector F n and the difference eigenvector D k Perform feature connection to form a comprehensive difference feature vector C,

[0093] C=[T1,T2,...,T m , F1, F2, ..., F n , D1, D2, ..., D k ].

[0094] Furthermore, to capture subtle differences, there may be subtle differences in the spectrum between AI synthesized audio signals and real audio signals, which may be ignored in conventional feature extraction methods. Through spectrum perturbation, these subtle differences can be amplified, allowing the detection model to more easily capture the characteristics of AI synthesized audio, thereby improving the accuracy of detection.

[0095] S4: According to steps S1 to S3, the real audio signal and the AI ​​synthesized audio signal are processed to form a feature vector set, and the difference feature vectors and corresponding labels in the feature vector set are divided into a training set and a test set.

[0096] Furthermore, the real audio and AI synthesized audio are labeled as corresponding categories, such as 0 or 1.

[0097] S5: Use the training set to train the neural network model, and use the test set to verify and test the trained neural network model, and finally realize AI speech detection. It should be noted that:

[0098] like Figure 5 As shown, the neural network model is a neural network model containing three fully connected layers, and ReLU is used as the activation function between each fully connected layer.

[0099] The neural network model provided by the present invention includes multiple fully connected layers (FC layers), ReLU activation function, Dropout layer and an output layer;

[0100] model.0.weight(512, 180): This is the weight matrix of the first fully connected layer, which has 512 input units and 180 output units.

[0101] model.0.bias(512): This is the bias term of the first fully connected layer.

[0102] model.3.weight(256,512): This is the weight matrix of the second fully connected layer, which has 256 input units and 512 output units.

[0103] model.3.bias(256): This is the bias term of the second fully connected layer.

[0104] model.6.weight(128,256): This is the weight matrix of the third fully connected layer, which has 128 input units and 256 output units.

[0105] model.6.bias(128): This is the bias term of the third fully connected layer.

[0106] model.9.weight(1,128) : This is the weight matrix of the fourth fully connected layer, which has 1 input unit and 128 output units.

[0107] model.9.bias(1): This is the bias term of the fourth fully connected layer.

[0108] During the back propagation process, the gradient is passed along the network from the output layer to the input layer. The key operations in this flowchart are:

[0109] AccumulateGrad: represents the gradient accumulation operation, which is used to update weight and bias parameters.

[0110] TBackwardO: represents the back propagation operation of a tensor.

[0111] AddmmBackwardO: represents the back propagation of matrix multiplication and addition operations.

[0112] ReluBackwardO: represents the back propagation operation of the ReLU activation function.

[0113] NativeDropoutBackwardO: represents the back propagation operation of the Dropout layer.

[0114] SigmoidBackwardO: represents the back propagation operation of the Sigmoid function.

[0115] Figure 5 It shows how the gradients are back-propagated through the layers to update the weights and biases of the model. During training, the gradient of the loss function is passed along this path to each layer in order to update the model parameters to minimize the loss function.

[0116] The neural network model is trained using the training set. During the training process, different hyperparameter settings can be tried to find the optimal configuration.

[0117] A second aspect of the present invention provides an AI synthesized sound detection system based on spectrum disturbance analysis, comprising:

[0118] A signal processing unit, used for collecting audio signals and preprocessing the audio signals to obtain standardized audio signals;

[0119] A difference feature vector acquisition unit is used to perform spectrum analysis and spectrum perturbation on the standardized audio signal, and calculate the original spectrum graph and the perturbed spectrum graph; and acquire the difference feature vector based on the original spectrum graph and the perturbed spectrum graph;

[0120] The speech detection unit is used to process the real audio signal and the AI ​​synthesized audio signal to form a feature vector set, and divide the difference feature vectors and corresponding labels in the feature vector set into a training set and a test set; use the training set to train the neural network model, and use the test set to verify and test the trained neural network model, and finally realize AI speech detection.

[0121] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0122] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0123] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0125] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A detection method for AI synthesized sound based on spectrum disturbance analysis, characterized in that: include: S1: collecting audio signals and preprocessing the audio signals to obtain standardized audio signals; S2: performing spectrum analysis and spectrum perturbation on the standardized audio signal, and calculating an original spectrum graph and a perturbed spectrum graph; S3: Obtaining a difference feature vector based on the original spectrum graph and the perturbed spectrum graph; S4: Processing the real audio signal and the AI ​​synthesized audio signal according to steps S1 to S3 to form a feature vector set, and dividing the difference feature vectors and corresponding labels in the feature vector set into a training set and a test set; S5: Use the training set to train the neural network model, use the test set to verify and test the trained neural network model, and finally realize AI voice detection.

2. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 1, characterized in that: The specific operations of preprocessing the audio signal include: Converting the sampling rate of the audio signal by downsampling, and adjusting the sampling rate of the audio signal to a target sampling rate; removing noise from the audio signal after the sampling rate adjustment; Amplitude normalization is performed on the audio signal after noise removal.

3. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 2, characterized in that: The calculation of the original spectrum diagram includes: The standardized audio signal is segmented and a window function is applied. The frequency component of each audio signal is obtained by fast Fourier transform, and the power spectrum density is calculated. The power spectrum density at each time point is arranged in chronological order to generate an original spectrum diagram showing the change of signal frequency over time.

4. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 3, characterized in that: The calculation of the perturbed spectrum diagram includes: Adding random noise as a disturbance to the original spectrogram, converting the disturbed spectrogram back into a time-domain audio signal, and finally calculating the disturbed spectrogram; Assuming the original spectrum is x(t), the noise amplitude factor is ∈, the calculation of the noise signal n(t) is: n(t)=∈·randn(t); Where t is any time; The noise signal is superimposed on the original spectrum to obtain the disturbed spectrum x'(t). x'(t)=x(t)+n(t).

5. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 4, characterized in that: The acquisition of the difference feature vector includes: The frequency domain features and time domain features of the original spectrum graph are extracted, and the spectrum difference features are extracted by comparing the original spectrum graph and the perturbed spectrum graph. The frequency domain features, the time domain features and the spectrum difference features are fused to form a difference feature vector.

6. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 5, characterized in that: The acquisition of the spectrum difference feature includes: The spectrum difference between the original spectrum graph and the spectrum graph after disturbance and the signal-to-noise ratio between the original signal and the signal after disturbance are fused by weighted synthesis to obtain the spectrum difference feature; The spectrum difference D spectrum The calculation includes, D spectrum =∑ f |P original (f)-P perturbed (f)|; Among them, P original (f) represents the spectrum of the original spectrogram, P perturbed represents the spectrum of the perturbed spectrum graph, and f represents the frequency; The signal-to-noise ratio SNR dB The calculation includes, Among them, Power signal Indicates signal power, Power noise Represents the noise power.

7. The method for detecting AI synthesized sound based on spectrum disturbance analysis according to claim 5, characterized in that: The formation of the difference feature vector specifically includes: The time domain feature vector T m , frequency domain feature vector F n and the difference eigenvector D k Perform feature connection to form a comprehensive difference feature vector C, C=[T1,T2,...,T m ,F1,F2,...,F n ,D1,D2,...,D k ]。 8. The method for detecting AI synthesized speech based on spectrum disturbance analysis according to any one of claims 1 to 7, characterized in that: The neural network model is a neural network model including three fully connected layers, and ReLU is used as an activation function between each fully connected layer.

9. A system using the AI ​​synthesized sound detection method based on spectrum disturbance analysis as described in any one of claims 1 to 8, characterized in that: include: A signal processing unit, used for collecting audio signals and preprocessing the audio signals to obtain standardized audio signals; A difference feature vector acquisition unit, used for performing spectrum analysis and spectrum perturbation on the standardized audio signal, and calculating an original spectrum graph and a perturbed spectrum graph; and acquiring a difference feature vector based on the original spectrum graph and the perturbed spectrum graph; The speech detection unit is used to process the real audio signal and the AI ​​synthesized audio signal to form a feature vector set, and divide the difference feature vectors and corresponding labels in the feature vector set into a training set and a test set; use the training set to train the neural network model, and use the test set to verify and test the trained neural network model, and finally realize AI speech detection.

Images