An audio recompression anti-forensics method based on generative adversarial network

Abstract

The present invention discloses an audio recompression anti-forensics method based on a generative adversarial network. In the training stage, the training set is first used to perform a separate network training on the generator for 5 rounds, and then the training set is used to perform adversarial training on the generator and discriminator for at least 30 rounds. The adversarial training is stopped after the generator and discriminator are adversarially trained for at least 30 rounds. Finally, the generator model and the discriminator model are trained to obtain the generator model and the parameters of the generator model, and the adversarial training is stopped in advance if the classification probability representing the true and the classification probability representing the false output of the discriminator in the current round are close to each other during the adversarial training process. In the testing stage, any decompressed audio obtained by recompression and then decompression is given as the test audio. The test audio is then tested using the generator model, and the generator model outputs the anti-forensic audio corresponding to the test audio. The advantage is that it can well erase the traces of recompression while ensuring good audio perception quality.

Description

Technical Field

[0001] The present invention relates to an anti-forensic technology, and in particular to an audio recompression anti-forensic method based on a generative adversarial network. Background Art

[0002] Anti-forensic technology is a new technology developed in recent years. Its purpose is to use appropriate post-processing operations to eliminate or hide traces of tampering operations, which can cause forensic technology to make misjudgments. Anti-forensic technology can reveal problems with the forensic technology itself, allowing forensics practitioners to repair and improve existing vulnerabilities or shortcomings, thereby improving the robustness of the forensic technology itself and its ability to resist attacks. Research on anti-forensic technology can promote the improvement of forensic technology through the mutual attack and defense and game between anti-forensic technology and forensic technology. This not only improves the security reputation of multimedia files, but also indirectly increases the cost of malicious tampering.

[0003] Criminals often forge multimedia files, such as digital audio files, for various purposes. This process inevitably involves recompression. If the digital audio file is in a lossy compression format (such as MP3, AAC, AMR, or OGG), the file must first be decoded to a waveform file, which is then tampered with. The modified waveform file is then re-encoded into the corresponding compressed format. This process is called recompression. Detecting the compression history of digital audio files can provide crucial evidence for audio forensics. Recompressed digital audio files leave behind statistical traces of the recompression, and erasing or concealing these traces is crucial for forensic analysis. Furthermore, when forging digital audio files using anti-forensic techniques, criminals also leave new traces of their manipulations within the files. These traces can be used as additional evidence to verify the originality and authenticity of the digital audio files.

[0004] The literature M.Mascia, A.Canclini, F.Antonacci, M.Tagliasacchi, A.Sarti, S.Tubaro.Forensic, anti-forensic of analysis of indoor / outdoor classifiersbased on acoustic clues.Proceedings of the 23rd European Signal Processing Conference, 2015. discusses in detail the game between audio anti-forensics and forensics. It modifies the power grid frequency to confuse the results of forensic technology. The disadvantage of this method is that it cannot be used for battery-powered recordings. Literature Hong Zhao, Yifan Chen, Rui Wang, Hafiz Malik. Anti-forensics of environmental-signature-based audio splicing detection, its countermeasure via rich-features classification. IEEE Transactions on Information Forensics and Security, 2016. (Anti-forensics of audio splicing detection based on environmental features, its countermeasure is through rich feature classification. IEEE Transactions on Information Forensics and Security, 2016.), which proposes an anti-forensics algorithm for splicing detection based on environmental signatures, by using a rich feature model composed of Fourier coefficients, spectral characteristics, high-order statistics of music noise residuals and modulation spectrum coefficients to capture the traces of dereverberation attacks, thereby achieving the anti-forensic attack effect. The proposed rich feature model can have a certain average detection accuracy for audio that has undergone anti-forensic operations, but the robustness of the method is poor.Literature Diqun Yan, Yongkang Gong, Tianyun Liu. Antiforensics of speech resampling using dual-path Strategy. Wireless Communications and Mobile Computing, 2021. (Yan Diqun, Gong Yongkang, Liu Tianyun. Speech resampling anti-forensics based on dual-path strategy. Wireless Communications and Mobile Computing, 2021.), it simultaneously modifies the low-frequency and high-frequency content of the digital audio signal. The attack on the low-frequency part uses a median filter to destroy the linear relationship between adjacent sampling points of the resampled audio signal, and the attack on the high-frequency part uses the addition of Gaussian perturbation noise to destroy the periodicity between the sampling points. The anti-forensics audio formed by the combination of the low-frequency part and the high-frequency part after the attack can not only effectively hide the periodic traces left by the resampling operation, but also maintain a high auditory quality. However, this method is only applicable to resampling anti-forensics and cannot be used in re-compressed anti-forensics scenarios.

[0005] Judging from the research results that have been published so far, there is little attention paid to audio recompression anti-forensic technology at home and abroad. Compared with the relatively mature audio forensics technology that has been initially applied to some specific cases, the research on the currently proposed audio recompression anti-forensic technology is still in its infancy, and the existing audio recompression anti-forensic methods find it difficult to erase the traces of recompression while maintaining the perceptual quality of the audio. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an audio recompression anti-forensics method based on a generative adversarial network, which can effectively erase the recompression traces while ensuring good audio perception quality.

[0007] The technical solution adopted by the present invention to solve the above technical problems is: an audio recompression anti-forensics method based on a generative adversarial network, including a training phase and a testing phase, characterized in that the specific process of the training phase is as follows:

[0008] Step 1_1: Construct a training set containing several uncompressed original audio files and decompressed audio files obtained by recompressing each original audio file and then decompressing it. Build a generator and a discriminator. The sampling rate of the original audio files and the decompressed audio files is 8K.

[0009] Step 1_2: Set the Adam optimizer to optimize and update the weights and parameters of the generator and discriminator, and set the learning rate of the generator to 1×10 -4 The learning rate of the discriminator is 1×10 -5 ;

[0010] Step 1_3: Use the training set to train the generator separately for 5 rounds: Input each decompressed audio in the training set into the generator for network training. After each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss and the generator loss function value, which are recorded as Loss. audi and Loss G , Loss G =α×Loss audi +β×Loss adv ; Among them, the sampling rate of the adversarial sample is 8K, Num samp Indicates the number of sampling points, Num samp =8000, 1≤i≤Num samp , the symbol “||” is the absolute value symbol, x org,i Represents each original audio x in the training set org The i-th sampling point, x' db,i Represents each decompressed audio x in the training set db Input to the generator for network training The corresponding adversarial sample x' output by the generator db The i-th sampling point, α represents the Loss audi The weight of β represents Loss adv The weight of the generator network is trained separately, α=1000 and β=0, Loss adv represents adversarial loss;

[0011] Step 1_4: After the generator network is trained for 5 rounds, the generator and discriminator are trained against each other using the training set for at least 30 rounds: each decompressed audio in the training set is input into the generator for network training. At the end of each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss, which is recorded as Loss. audi , Then, the adversarial sample corresponding to each decompressed audio in the training set is input into the discriminator for network training. Similarly, each original audio in the training set is input into the discriminator for network training. At the end of each round of network training, the discriminator outputs a true classification probability and a false classification probability corresponding to each adversarial sample, as well as a true classification probability and a false classification probability corresponding to each original audio, and calculates the adversarial loss, which is recorded as Loss adv , After each round of adversarial training, the loss function value of the discriminator is calculated first, recorded as Loss D , Then calculate the loss function value of the generator, recorded as Loss G , LossG =α×Loss audi +β×Loss adv ; Where D() represents the discriminator, when j=1, D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the true classification probability, when j = 2 D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the false classification probability, when j = 1 D(x org ) j Represents each original audio x in the training set org Input to the discriminator for network training The discriminator output represents the true classification probability. When j = 2, D(x org ) j Represents each original audio x in the training set org The output of the discriminator represents the false classification probability when input to the discriminator for network training. During the adversarial training of the generator and the discriminator, α = 1000 and β = 1. The iteration ratio of the generator and the discriminator in the adversarial training is set to 1:5.

[0012] Step 1_5: After the generator and discriminator have been trained adversarially for at least 30 rounds, stop the adversarial training. Finally, the generator model and the discriminator model are trained and the parameters of the generator model are saved. During the adversarial training process, if the probability of the classification representing the true and the probability of the classification representing the false output of the discriminator in the current round are close, then stop the adversarial training early.

[0013] The specific process of the testing phase is as follows:

[0014] Any decompressed audio obtained by recompressing and then decompressing is given as the test audio; then the test audio is tested using the generator model, and the generator model outputs the anti-forensic audio corresponding to the test audio.

[0015] The generator includes an input module, a U-Net module, and an output module. The input module is mainly composed of a first convolutional layer and a second convolutional layer connected in sequence. The output module is mainly composed of a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer connected in sequence. The U-Net module is composed of an encoder that uses 7 downsampling operations to obtain high-level semantic information and a decoder that uses 7 upsampling operations to restore resolution. The input end of the first convolutional layer receives an audio with a sampling rate of 8K as the input end of the generator, the input end of the second convolutional layer receives the features output by the output end of the first convolutional layer, the input end of the encoder receives the features output by the output end of the second convolutional layer, and the decoder is used to restore the resolution. The input end of the three convolutional layers receives the features obtained by concatenating the features output by the output end of the second convolutional layer and the features output by the output end of the decoder. The input end of the fourth convolutional layer receives the features output by the output end of the third convolutional layer. The input end of the fifth convolutional layer receives the features output by the output end of the fourth convolutional layer. The output end of the fifth convolutional layer serves as the output end of the generator, and the output end of the fifth convolutional layer outputs an audio with a sampling rate of 8K. Among them, the convolution kernel size of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are all 1×30, the activation function is ReLU, and the step size is 2. The convolution kernel size of the fifth convolutional layer is 1×1, and the step size is 1.

[0016] The encoder includes seven maximum pooling layers, from the first maximum pooling layer to the seventh maximum pooling layer, and seven convolution groups, from the first convolution group to the seventh convolution group, respectively. The decoder includes seven deconvolution layers, from the first deconvolution layer to the seventh deconvolution layer, respectively, and seven convolution groups, from the eighth convolution group to the fourteenth convolution group, respectively. The input end of the first maximum pooling layer receives the features output by the output end of the second convolution layer as the input end of the encoder, the input end of the first convolution group receives the features output by the output end of the first maximum pooling layer, the input end of the second maximum pooling layer receives the features output by the output end of the first convolution group, and the input end of the second convolution group receives the features output by the output end of the second maximum pooling layer. The input end of the 3rd maximum pooling layer receives the features output by the output end of the 2nd convolution group, the input end of the 3rd convolution group receives the features output by the output end of the 3rd maximum pooling layer, the input end of the 4th maximum pooling layer receives the features output by the output end of the 3rd convolution group, the input end of the 4th convolution group receives the features output by the output end of the 4th maximum pooling layer, the input end of the 5th maximum pooling layer receives the features output by the output end of the 4th convolution group, the input end of the 5th convolution group receives the features output by the output end of the 5th maximum pooling layer, the input end of the 6th maximum pooling layer receives the features output by the output end of the 5th convolution group, and the input end of the 6th convolution group receives the features output by the output end of the 6th maximum pooling layer. The input end of the 7th maximum pooling layer receives the features output by the output end of the 6th convolution group, the input end of the 7th convolution group receives the features output by the output end of the 7th maximum pooling layer, the input end of the 8th convolution group receives the features output by the output end of the 7th convolution group, the input end of the 1st deconvolution layer receives the features output by the output end of the 8th convolution group, the input end of the 9th convolution group receives the features obtained by concatenating the features output by the output end of the 1st deconvolution layer and the features output by the output end of the 6th convolution group, the input end of the 2nd deconvolution layer receives the features output by the output end of the 9th convolution group, and the input end of the 10th convolution group receives the features output by the output end of the 2nd deconvolution layer. The input end of the 3rd deconvolution layer receives the feature output by the output end of the 10th convolution group, the input end of the 11th convolution group receives the feature obtained by splicing the feature output by the output end of the 3rd deconvolution layer and the feature output by the output end of the 4th convolution group, the input end of the 4th deconvolution layer receives the feature output by the output end of the 11th convolution group, the input end of the 12th convolution group receives the feature obtained by splicing the feature output by the output end of the 4th deconvolution layer and the feature output by the output end of the 3rd convolution group, and the input end of the 5th deconvolution layer receives the feature output by the output end of the 12th convolution group.The input of the 13th convolution group receives the features obtained by concatenating the output of the 5th deconvolution layer with the features obtained by concatenating the output of the 2nd convolution group. The input of the 6th deconvolution layer receives the features obtained by concatenating the output of the 13th convolution group. The input of the 14th convolution group receives the features obtained by concatenating the output of the 6th deconvolution layer with the features obtained by concatenating the output of the 1st convolution group. The input of the 7th deconvolution layer receives the features obtained by concatenating the output of the 14th convolution group. The output of the 7th deconvolution layer serves as the output of the decoder. The convolution kernel size of the 1st to 7th maximum pooling layers is 1×30 with a stride of 2. The convolution kernel size of the 1st to 7th deconvolution layers is 1×30 with a stride of 1. The features are downsampled through the maximum pooling layer, and upsampled through the deconvolution layer.

[0017] The structures of the 1st to 6th convolution groups and the 9th to 14th convolution groups are the same. They are all composed of the sixth convolution layer and the seventh convolution layer connected in sequence. The input end of the sixth convolution layer serves as the input end of the convolution group to which it belongs, and the input end of the seventh convolution layer receives the features output by the output end of the sixth convolution layer, and the output end of the seventh convolution layer serves as the output end of the convolution group to which it belongs; the structures of the 7th and 8th convolution groups are the same. They are all composed of the eighth convolution layer, and the input end of the eighth convolution layer serves as the input end of the convolution group to which it belongs, and the output end of the eighth convolution layer serves as the output end of the convolution group to which it belongs; among them, the convolution kernel size of the 6th, 7th and 8th convolution layers is 1×30, the activation function is ReLU, and the step size is 2.

[0018] The discriminator includes the 1st to 8th residual blocks, and a classification module composed of a fully connected layer and a Softmax activation layer. The 1st to 8th residual blocks have the same functions. The input end of the 1st residual block receives an audio with a sampling rate of 8K as the input end of the discriminator. The input end of the 2nd residual block receives the result of adding the features output from the output end of the 1st residual block to the audio input to the input end of the 1st residual block. The input end of the 3rd residual block receives the result of adding the features output from the output end of the 2nd residual block to the features input to the input end of the 2nd residual block. The input end of the 4th residual block receives the result of adding the features output from the output end of the 3rd residual block to the features input to the input end of the 3rd residual block. The input end of the 5th residual block receives the result of adding the features output from the output end of the 4th residual block to the features input to the input end of the 4th residual block. The input end of the 6th residual block receives the result of adding the features output by the output end of the 5th residual block to the features input to the input end of the 5th residual block, the input end of the 7th residual block receives the result of adding the features output by the output end of the 6th residual block to the features input to the input end of the 6th residual block, the input end of the 8th residual block receives the result of adding the features output by the output end of the 7th residual block to the features input to the input end of the 7th residual block, the input end of the fully connected layer receives the result of adding the features output by the output end of the 8th residual block to the features input to the input end of the 8th residual block, the input end of the Softmax activation layer receives the output value output by the output end of the fully connected layer, and the output end of the Softmax activation layer outputs a true classification probability and a false classification probability as the output end of the discriminator; among them, the output value of the fully connected layer is 2.

[0019] The structures of the first to eighth residual blocks are the same. They are all composed of the ninth convolutional layer, batch normalization layer, and ReLU activation layer connected in sequence. The input of the ninth convolutional layer is the input of the residual block where it is located, the input of the batch normalization layer receives the features output by the output of the ninth convolutional layer, the input of the ReLU activation layer receives the features output by the output of the batch normalization layer, and the output of the ReLU activation layer is the output of the residual block where it is located. Among them, the convolution kernel size of the ninth convolutional layer in the first residual block is 1×30, the number of convolution kernels is 8, and the stride is 2. The convolution kernel size of the ninth convolutional layer in the second residual block is 1×30, the number of convolution kernels is 16, and the stride is 2. The third residual block The convolution kernel size of the ninth convolution layer in the block is 1×30, the number of convolution kernels is 32, and the stride is 2. The convolution kernel size of the ninth convolution layer in the fourth residual block is 1×30, the number of convolution kernels is 64, and the stride is 2. The convolution kernel size of the ninth convolution layer in the fifth residual block is 1×30, the number of convolution kernels is 128, and the stride is 2. The convolution kernel size of the ninth convolution layer in the sixth residual block is 1×30, the number of convolution kernels is 256, and the stride is 2. The convolution kernel size of the ninth convolution layer in the seventh residual block is 1×30, the number of convolution kernels is 512, and the stride is 2. The convolution kernel size of the ninth convolution layer in the first residual block is 1×30, the number of convolution kernels is 1024, and the stride is 2.

[0020] The heavy compression adopts AMR compression.

[0021] Compared with the prior art, the advantages of the present invention are:

[0022] 1) Through the game principle of the generative adversarial network composed of a generator and a discriminator, the statistical distribution of the original audio is learned, a reasonable loss function is designed, and targeted modifications are made to the heavily compressed audio. The resulting trained generator model can effectively remove the traces of heavy compression while ensuring good speech quality.

[0023] 2) Since the main advantage of the generative adversarial network is that after iterative training, the generator can obtain samples with better performance, the method of the present invention uses the discriminator as a simulated recompression forensic detector to design a corresponding adversarial framework. The discriminator is used to distinguish between original uncompressed audio and recompressed audio, thereby prompting the generator to generate high-quality audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a block diagram of the overall implementation of the training phase of the method of the present invention;

[0025] Figure 2 Schematic diagram of the overall implementation of the generator in the method of the present invention;

[0026] Figure 3Schematic diagram of the overall implementation of the U-Net module in the generator in the method of the present invention;

[0027] Figure 4 Schematic diagram of the overall implementation of the discriminator in the method of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0029] The present invention proposes an audio recompression anti-forensics method based on generative adversarial network, which includes a training phase and a testing phase. Figure 1 As shown, the specific process of the training phase is as follows:

[0030] Step 1_1: Construct a training set containing several uncompressed original audio files and decompressed audio files obtained by recompressing each original audio file and then decompressing it. Build a generator and a discriminator. In the specific implementation, the training set directly uses an existing dataset, and the sampling rate of the original audio and decompressed audio is 8K.

[0031] Step 1_2: Set the Adam optimizer to optimize and update the weights and parameters of the generator and discriminator, and set the learning rate of the generator to 1×10 -4 The learning rate of the discriminator is 1×10 -5 .

[0032] Step 1_3: Use the training set to train the generator separately for 5 rounds: Input each decompressed audio in the training set into the generator for network training. After each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss and the generator loss function value, which are recorded as Loss. audi and Loss G , Loss G =α×Loss audi +β×Loss adv ; Among them, the sampling rate of the adversarial sample is 8K, Num samp Indicates the number of sampling points, Num samp =8000, 1≤i≤Num samp , the symbol “||” is the absolute value symbol, x org,i Represents each original audio x in the training set org The i-th sampling point, x' db,i Represents each decompressed audio x in the training set db Input to the generator for network training The corresponding adversarial sample x' output by the generator db The i-th sampling point, x' db =G(xdb ), G() represents the generator, α represents the Loss audi The weight of β represents Loss adv The weight of the generator network is trained separately, α=1000 and β=0, Loss adv represents the adversarial loss.

[0033] Step 1_4: After the generator network is trained for 5 rounds, the generator and discriminator are trained against each other using the training set for at least 30 rounds: each decompressed audio in the training set is input into the generator for network training. At the end of each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss, which is recorded as Loss. audi , Then, the adversarial sample corresponding to each decompressed audio in the training set is input into the discriminator for network training. Similarly, each original audio in the training set is input into the discriminator for network training. At the end of each round of network training, the discriminator outputs a true classification probability and a false classification probability corresponding to each adversarial sample, as well as a true classification probability and a false classification probability corresponding to each original audio, and calculates the adversarial loss, which is recorded as Loss adv , After each round of adversarial training, the loss function value of the discriminator is calculated first, recorded as Loss D , Then calculate the loss function value of the generator, recorded as Loss G , Loss G =α×Loss audi +β×Loss adv ; Where D() represents the discriminator, when j=1, D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the true classification probability, when j = 2 D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the false classification probability, when j = 1 D(x org ) j Represents each original audio x in the training set org Input to the discriminator for network training The discriminator output represents the true classification probability. When j = 2, D(x org ) j Represents each original audio x in the training set orgThe input to the discriminator for network training is the output of the discriminator, which represents the false classification probability. During the adversarial training of the generator and the discriminator, α = 1000 and β = 1. Because multiple training of the discriminator can make the generator train better, the iteration ratio of the generator and the discriminator in the adversarial training is set to 1:5.

[0034] Since the attack should not leave any auditory perceptible traces, the proposed method uses the mean absolute difference between the original audio and the generated adversarial samples to simulate the auditory perceptual loss, i.e.

[0035] The purpose of the method of the present invention is to eliminate the trace artifacts caused by the compression operation on the audio, and to design a reasonable loss function to improve the quality of the generated audio. The goal is to add as little disturbance as possible, and finally generate audio that eliminates the compression traces while improving the quality of the generated audio. Therefore, the loss function of the generator is defined as Loss G =α×Loss audi +β×Loss adv .

[0036] Adversarial loss can enable the generator to be better trained to fool the discriminator.

[0037] The discriminator is used as the opponent of the generator. In the iterative training of the two, the generator tries to forge audio to avoid being identified by the discriminator. The purpose of the discriminator is to simulate the audio recompression detection algorithm to detect whether the audio has been compressed once or recompressed. Therefore, the loss function of the discriminator is defined as

[0038] Step 1_5: After the generator and discriminator have been trained adversarially for at least 30 rounds, stop the adversarial training. Finally, the generator model and the discriminator model are trained and the parameters of the generator model are saved. During the adversarial training process, if the probability of the classification representing the true and the probability of the classification representing the false output of the discriminator in the current round are close, then stop the adversarial training early. In the ideal state, the probability of the classification representing the true and the probability of the classification representing the false are both 0.5, but in reality, it does not reach 0.5.

[0039] The specific process of the testing phase is as follows:

[0040] A decompressed audio obtained by recompressing and then decompressing is arbitrarily given as the test audio; the test audio is then tested using the generator model, and the generator model outputs the anti-forensic audio corresponding to the test audio, in which the re-compression traces are erased.

[0041] In a specific embodiment, Figure 2As shown, the generator includes an input module, a U-Net module, and an output module. The input module is mainly composed of the first convolutional layer and the second convolutional layer connected in sequence. The output module is mainly composed of the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer connected in sequence. The U-Net module is composed of an encoder that uses 7 downsampling operations to obtain high-level semantic information and a decoder that uses 7 upsampling operations to restore resolution. The input end of the first convolutional layer receives an audio with a sampling rate of 8K as the input end of the generator, that is, the decompressed audio obtained by recompressing the original audio. The input end of the second convolutional layer receives the features output by the output end of the first convolutional layer, the input end of the encoder receives the features output by the output end of the second convolutional layer, and the input end of the third convolutional layer receives the features output by the output end of the second convolutional layer. The features obtained by concatenating the features output by the output end of the second convolutional layer and the features output by the output end of the decoder are received, the features output by the output end of the third convolutional layer are received at the input end of the fourth convolutional layer, the features output by the output end of the fifth convolutional layer are received at the input end of the fifth convolutional layer, the output end of the fifth convolutional layer is used as the output end of the generator, and the output end of the fifth convolutional layer outputs an audio with a sampling rate of 8K, that is, an adversarial sample corresponding to the decompressed audio; wherein, the convolution kernel size of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are all 1×30, the activation function is ReLU, and the stride is 2, the convolution kernel size of the fifth convolutional layer is 1×1, and the stride is 1. The splicing operation is a conventional operation method in neural networks.

[0042] In a specific embodiment, Figure 3As shown, the encoder includes seven maximum pooling layers from the 1st maximum pooling layer to the 7th maximum pooling layer, and seven convolution groups from the 1st convolution group to the 7th convolution group, a total of seven convolution groups. The decoder includes seven deconvolution layers from the 1st deconvolution layer to the 7th deconvolution layer, a total of seven deconvolution layers, and seven convolution groups from the 8th convolution group to the 14th convolution group, a total of seven convolution groups. The input end of the first maximum pooling layer receives the features output by the output end of the second convolution layer as the input end of the encoder, the input end of the first convolution group receives the features output by the output end of the first maximum pooling layer, the input end of the second maximum pooling layer receives the features output by the output end of the first convolution group, and the input end of the second convolution group receives the features output by the output end of the second maximum pooling layer. The input end of the 3rd maximum pooling layer receives the features output by the output end of the 2nd convolution group, the input end of the 3rd convolution group receives the features output by the output end of the 3rd maximum pooling layer, the input end of the 4th maximum pooling layer receives the features output by the output end of the 3rd convolution group, the input end of the 4th convolution group receives the features output by the output end of the 4th maximum pooling layer, the input end of the 5th maximum pooling layer receives the features output by the output end of the 4th convolution group, the input end of the 5th convolution group receives the features output by the output end of the 5th maximum pooling layer, the input end of the 6th maximum pooling layer receives the features output by the output end of the 5th convolution group, and the input end of the 6th convolution group receives the features output by the output end of the 6th maximum pooling layer. The input end of the 7th maximum pooling layer receives the features output by the output end of the 6th convolution group, the input end of the 7th convolution group receives the features output by the output end of the 7th maximum pooling layer, the input end of the 8th convolution group receives the features output by the output end of the 7th convolution group, the input end of the 1st deconvolution layer receives the features output by the output end of the 8th convolution group, the input end of the 9th convolution group receives the features obtained by concatenating the features output by the output end of the 1st deconvolution layer and the features output by the output end of the 6th convolution group, the input end of the 2nd deconvolution layer receives the features output by the output end of the 9th convolution group, and the input end of the 10th convolution group receives the features output by the output end of the 2nd deconvolution layer. The input end of the 3rd deconvolution layer receives the feature output by the output end of the 10th convolution group, the input end of the 11th convolution group receives the feature obtained by splicing the feature output by the output end of the 3rd deconvolution layer and the feature output by the output end of the 4th convolution group, the input end of the 4th deconvolution layer receives the feature output by the output end of the 11th convolution group, the input end of the 12th convolution group receives the feature obtained by splicing the feature output by the output end of the 4th deconvolution layer and the feature output by the output end of the 3rd convolution group, and the input end of the 5th deconvolution layer receives the feature output by the output end of the 12th convolution group.The input end of the 13th convolution group receives the features obtained by concatenating the features output by the output end of the 5th deconvolution layer and the features output by the output end of the 2nd convolution group. The input end of the 6th deconvolution layer receives the features output by the output end of the 13th convolution group. The input end of the 14th convolution group receives the features obtained by concatenating the features output by the output end of the 6th deconvolution layer and the features output by the output end of the 1st convolution group. The input end of the 7th deconvolution layer receives the features output by the output end of the 14th convolution group. The output end of the 7th deconvolution layer serves as the output end of the decoder. Among them, the convolution kernel size of the 1st to 7th maximum pooling layers is 1×30 and the stride is 2. The convolution kernel size of the 1st to 7th deconvolution layers is 1×30 and the stride is 1. The features are down-sampled through the maximum pooling layer, and the features are up-sampled through the deconvolution layer. The concatenation operation is a common operation in neural networks. The two main features of the U-Net module are its U-shaped structure and skip connections. The U-Net module's encoder performs seven downsampling operations to obtain high-level semantic information, while the decoder performs seven corresponding upsampling operations to restore resolution. Skip connections are introduced to reduce the spatial information loss caused by downsampling and to reconstruct audio details. Through concatenation, the upsampled features contain more low-level semantic information, resulting in a more refined result.

[0043] In a specific embodiment, Figure 3 As shown in the figure, the structures of the 1st to 6th convolution groups and the 9th to 14th convolution groups are the same. They are all composed of the sixth convolution layer and the seventh convolution layer connected in sequence. The input end of the sixth convolution layer serves as the input end of the convolution group to which it is located, and the input end of the seventh convolution layer receives the features output by the output end of the sixth convolution layer, and the output end of the seventh convolution layer serves as the output end of the convolution group to which it is located; the structures of the 7th and 8th convolution groups are the same. They are all composed of the eighth convolution layer, and the input end of the eighth convolution layer serves as the input end of the convolution group to which it is located, and the output end of the eighth convolution layer serves as the output end of the convolution group to which it is located; among them, the convolution kernel size of the 6th convolution layer, the 7th convolution layer, and the 8th convolution layer are all 1×30, the activation function is ReLU, and the step size is 2.

[0044] In a specific embodiment, Figure 4As shown, the discriminator includes the 1st residual block to the 8th residual block, and a classification module composed of a fully connected layer and a softmax activation layer. The functions of the 1st residual block to the 8th residual block are the same. The input end of the 1st residual block receives an audio with a sampling rate of 8K, that is, an adversarial sample or original audio, as the input end of the discriminator. The input end of the 2nd residual block receives the result of adding the features output by the output end of the 1st residual block to the audio input to the input end of the 1st residual block. The input end of the 3rd residual block receives the result of adding the features output by the output end of the 2nd residual block to the features input to the input end of the 2nd residual block. The input end of the 4th residual block receives the result of adding the features output by the output end of the 3rd residual block to the features input to the input end of the 3rd residual block. The input end of the 5th residual block receives the result of adding the features output by the output end of the 4th residual block to the features input to the input end of the 4th residual block. The input end of the 6th residual block receives the result of adding the features output by the output end of the 5th residual block to the features input to the input end of the 5th residual block. The input end of the 7th residual block receives the result of adding the features output by the output end of the 6th residual block to the features input to the input end of the 6th residual block. The input end of the 8th residual block receives the result of adding the features output by the output end of the 7th residual block to the features input to the input end of the 7th residual block. The input end of the fully connected layer receives the result of adding the features output by the output end of the 8th residual block to the features input to the input end of the 8th residual block. The input end of the Softmax activation layer receives the output value output by the output end of the fully connected layer. The output end of the Softmax activation layer serves as the output end of the discriminator and outputs a true classification probability and a false classification probability. Among them, the fully connected layer has two output values, and the addition operation is a common operation method in neural networks.

[0045] In a specific embodiment, Figure 4As shown in FIG, the structures of the first residual block to the eighth residual block are the same. They are all composed of a ninth convolutional layer, a batch normalization layer (Batch Normalization), and a ReLU activation layer connected in sequence. The input end of the ninth convolutional layer is the input end of the residual block in which it is located, the input end of the batch normalization layer receives the features output by the output end of the ninth convolutional layer, the input end of the ReLU activation layer receives the features output by the output end of the batch normalization layer, and the output end of the ReLU activation layer is the output end of the residual block in which it is located. Among them, the convolution kernel size of the ninth convolutional layer in the first residual block is 1×30, the number of convolution kernels is 8, and the stride is 2. The convolution kernel size of the ninth convolutional layer in the second residual block is 1×30, the number of convolution kernels is 16, and the stride is 2. The convolution kernel size of the ninth convolutional layer in the third residual block is large. The convolution kernel size of the ninth convolution layer in the 4th residual block is 1×30, the number of convolution kernels is 64, and the step size is 2. The convolution kernel size of the ninth convolution layer in the 5th residual block is 1×30, the number of convolution kernels is 128, and the step size is 2. The convolution kernel size of the ninth convolution layer in the 6th residual block is 1×30, the number of convolution kernels is 256, and the step size is 2. The convolution kernel size of the ninth convolution layer in the 7th residual block is 1×30, the number of convolution kernels is 512, and the step size is 2. The convolution kernel size of the ninth convolution layer in the 1st residual block is 1×30, the number of convolution kernels is 1024, and the step size is 2.

[0046] As mentioned above, recompression uses AMR (Adaptive Multi-Rate). All original audio in the training set is recompressed using the same recompression method, such as AMR, and then decompressed to obtain the corresponding decompressed audio. Similarly, the test audio is also the decompressed audio obtained by recompressing the original audio using AMR and then decompressing it.

[0047] The method of the present invention trains the generator by using a large amount of data in the training phase, so that the generator learns the statistical distribution of the original audio; in the adversarial training between the discriminator and the generator, the optimization and update of the generator are promoted; finally, the trained generator model is extracted; in the testing phase, the anti-forensic audio obtained by the generator model can restore the quality of the re-compressed audio while eliminating or hiding the compression traces left by the re-compression process, thereby avoiding the detection of the forensic algorithm.

[0048] The evidence collection scenario in which the method of the present invention is used is: the evidence collector only accepts the audio to be tested in compressed format. After the evidence collector decompresses the audio to be tested, the method of the present invention is used to judge the heavy compression traces.

Claims

1. An audio recompression anti-forensic method based on generative adversarial network, including training phase and testing phase, characterized by The specific process of the training phase is as follows: Step 1_1: Construct a training set containing several uncompressed original audio files and decompressed audio files obtained by recompressing each original audio file and then decompressing it. Build a generator and a discriminator. The sampling rate of the original audio files and the decompressed audio files is 8K. Step 1_2: Set the Adam optimizer to optimize and update the weights and parameters of the generator and discriminator, and set the learning rate of the generator to 1×10 -4 The learning rate of the discriminator is 1×10 -5 ; Step 1_3: Use the training set to train the generator separately for 5 rounds: Input each decompressed audio in the training set into the generator for network training. After each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss and the generator loss function value, which are recorded as Loss. audi and Loss G , Loss G =α×Loss audi +β×Loss adv ; Among them, the sampling rate of the adversarial sample is 8K, Num samp Indicates the number of sampling points, Num samp =8000, 1≤i≤Num samp , the symbol "||" is the absolute value symbol, x org,i Represents each original audio x in the training set org The i-th sampling point, x' db,i Represents each decompressed audio x in the training set db Input to the generator for network training The corresponding adversarial sample x' output by the generator db The i-th sampling point, α represents the Loss audi The weight of β represents Loss adv The weight of the generator network is trained separately, α=1000 and β=0, Loss adv represents adversarial loss; Step 1_4: After the generator network is trained for 5 rounds, the generator and discriminator are trained against each other using the training set for at least 30 rounds: each decompressed audio in the training set is input into the generator for network training. At the end of each round of network training, the generator outputs the corresponding adversarial sample and calculates the auditory perception loss, which is recorded as Loss. audi , Then, the adversarial sample corresponding to each decompressed audio in the training set is input into the discriminator for network training. Similarly, each original audio in the training set is input into the discriminator for network training. At the end of each round of network training, the discriminator outputs a true classification probability and a false classification probability corresponding to each adversarial sample, as well as a true classification probability and a false classification probability corresponding to each original audio, and calculates the adversarial loss, which is recorded as Loss adv , After each round of adversarial training, the loss function value of the discriminator is calculated first, recorded as Loss D , Then calculate the loss function value of the generator, recorded as Loss G , Loss G =α×Loss audi +β×Loss adv ; Where D() represents the discriminator, when j=1, D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the true classification probability, when j = 2 D(x' db ) j Represents each decompressed audio x in the training set db The corresponding adversarial sample x' db Input to the discriminator for network training The discriminator output represents the false classification probability, when j = 1 D(x org ) j Represents each original audio x in the training set org Input to the discriminator for network training The discriminator output represents the true classification probability. When j = 2, D(x org ) j Represents each original audio x in the training set org The output of the discriminator represents the false classification probability when input to the discriminator for network training. During the adversarial training of the generator and the discriminator, α = 1000 and β = 1. The iteration ratio of the generator and the discriminator in the adversarial training is set to 1:

5. Step 1_5: After the generator and discriminator have been trained adversarially for at least 30 rounds, stop the adversarial training. Finally, the generator model and the discriminator model are trained and the parameters of the generator model are saved. During the adversarial training process, if the probability of the classification representing the true and the probability of the classification representing the false output of the discriminator in the current round are close, then stop the adversarial training early. The specific process of the testing phase is as follows: Any decompressed audio obtained by recompressing and then decompressing is given as the test audio; then the test audio is tested using the generator model, and the generator model outputs the anti-forensic audio corresponding to the test audio.

2. The method for anti-forensic analysis of audio recompression based on generative adversarial networks according to claim 1 is characterized in that The generator includes an input module, a U-Net module, and an output module. The input module is mainly composed of a first convolutional layer and a second convolutional layer connected in sequence. The output module is mainly composed of a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer connected in sequence. The U-Net module is composed of an encoder that uses 7 downsampling operations to obtain high-level semantic information and a decoder that uses 7 upsampling operations to restore resolution. The input end of the first convolutional layer receives an audio with a sampling rate of 8K as the input end of the generator, the input end of the second convolutional layer receives the features output by the output end of the first convolutional layer, the input end of the encoder receives the features output by the output end of the second convolutional layer, and the decoder is used to restore the resolution. The input end of the three convolutional layers receives the features obtained by concatenating the features output by the output end of the second convolutional layer and the features output by the output end of the decoder. The input end of the fourth convolutional layer receives the features output by the output end of the third convolutional layer. The input end of the fifth convolutional layer receives the features output by the output end of the fourth convolutional layer. The output end of the fifth convolutional layer serves as the output end of the generator, and the output end of the fifth convolutional layer outputs an audio with a sampling rate of 8K. Among them, the convolution kernel size of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are all 1×30, the activation function is ReLU, and the step size is 2. The convolution kernel size of the fifth convolutional layer is 1×1, and the step size is 1.

3. The method for anti-forensic analysis of audio recompression based on generative adversarial networks according to claim 2 is characterized in that The encoder includes seven maximum pooling layers, from the first maximum pooling layer to the seventh maximum pooling layer, and seven convolution groups, from the first convolution group to the seventh convolution group, respectively. The decoder includes seven deconvolution layers, from the first deconvolution layer to the seventh deconvolution layer, respectively, and seven convolution groups, from the eighth convolution group to the fourteenth convolution group, respectively. The input end of the first maximum pooling layer receives the features output by the output end of the second convolution layer as the input end of the encoder, the input end of the first convolution group receives the features output by the output end of the first maximum pooling layer, the input end of the second maximum pooling layer receives the features output by the output end of the first convolution group, and the input end of the second convolution group receives the features output by the output end of the second maximum pooling layer. The input end of the 3rd maximum pooling layer receives the features output by the output end of the 2nd convolution group, the input end of the 3rd convolution group receives the features output by the output end of the 3rd maximum pooling layer, the input end of the 4th maximum pooling layer receives the features output by the output end of the 3rd convolution group, the input end of the 4th convolution group receives the features output by the output end of the 4th maximum pooling layer, the input end of the 5th maximum pooling layer receives the features output by the output end of the 4th convolution group, the input end of the 5th convolution group receives the features output by the output end of the 5th maximum pooling layer, the input end of the 6th maximum pooling layer receives the features output by the output end of the 5th convolution group, and the input end of the 6th convolution group receives the features output by the output end of the 6th maximum pooling layer. The input end of the 7th maximum pooling layer receives the features output by the output end of the 6th convolution group, the input end of the 7th convolution group receives the features output by the output end of the 7th maximum pooling layer, the input end of the 8th convolution group receives the features output by the output end of the 7th convolution group, the input end of the 1st deconvolution layer receives the features output by the output end of the 8th convolution group, the input end of the 9th convolution group receives the features obtained by concatenating the features output by the output end of the 1st deconvolution layer and the features output by the output end of the 6th convolution group, the input end of the 2nd deconvolution layer receives the features output by the output end of the 9th convolution group, and the input end of the 10th convolution group receives the features output by the output end of the 2nd deconvolution layer. The input end of the 3rd deconvolution layer receives the feature output by the output end of the 10th convolution group, the input end of the 11th convolution group receives the feature obtained by splicing the feature output by the output end of the 3rd deconvolution layer and the feature output by the output end of the 4th convolution group, the input end of the 4th deconvolution layer receives the feature output by the output end of the 11th convolution group, the input end of the 12th convolution group receives the feature obtained by splicing the feature output by the output end of the 4th deconvolution layer and the feature output by the output end of the 3rd convolution group, and the input end of the 5th deconvolution layer receives the feature output by the output end of the 12th convolution group.The input of the 13th convolution group receives the features obtained by concatenating the output of the 5th deconvolution layer with the features obtained by concatenating the output of the 2nd convolution group. The input of the 6th deconvolution layer receives the features obtained by concatenating the output of the 13th convolution group. The input of the 14th convolution group receives the features obtained by concatenating the output of the 6th deconvolution layer with the features obtained by concatenating the output of the 1st convolution group. The input of the 7th deconvolution layer receives the features obtained by concatenating the output of the 14th convolution group. The output of the 7th deconvolution layer serves as the output of the decoder. The convolution kernel size of the 1st to 7th maximum pooling layers is 1×30 and the stride is 2. The convolution kernel size of the 1st to 7th deconvolution layers is 1×30 and the stride is 1. The features are down-sampled through the maximum pooling layer, and up-sampled through the deconvolution layer.

4. The method for anti-forensic analysis of audio recompression based on generative adversarial networks according to claim 3 is characterized in that The structures of the 1st to 6th convolution groups and the 9th to 14th convolution groups are the same. They are all composed of the sixth convolution layer and the seventh convolution layer connected in sequence. The input end of the sixth convolution layer serves as the input end of the convolution group to which it belongs, and the input end of the seventh convolution layer receives the features output by the output end of the sixth convolution layer, and the output end of the seventh convolution layer serves as the output end of the convolution group to which it belongs; the structures of the 7th and 8th convolution groups are the same. They are all composed of the eighth convolution layer, and the input end of the eighth convolution layer serves as the input end of the convolution group to which it belongs, and the output end of the eighth convolution layer serves as the output end of the convolution group to which it belongs; among them, the convolution kernel size of the 6th, 7th and 8th convolution layers is 1×30, the activation function is ReLU, and the step size is 2.

5. The method for anti-forensic analysis of audio recompression based on a generative adversarial network according to any one of claims 2 to 4, characterized in that The discriminator includes the 1st to 8th residual blocks, and a classification module composed of a fully connected layer and a Softmax activation layer. The 1st to 8th residual blocks have the same functions. The input end of the 1st residual block receives an audio with a sampling rate of 8K as the input end of the discriminator. The input end of the 2nd residual block receives the result of adding the features output from the output end of the 1st residual block to the audio input to the input end of the 1st residual block. The input end of the 3rd residual block receives the result of adding the features output from the output end of the 2nd residual block to the features input to the input end of the 2nd residual block. The input end of the 4th residual block receives the result of adding the features output from the output end of the 3rd residual block to the features input to the input end of the 3rd residual block. The input end of the 5th residual block receives the result of adding the features output from the output end of the 4th residual block to the features input to the input end of the 4th residual block. The input end of the 6th residual block receives the result of adding the features output by the output end of the 5th residual block to the features input to the input end of the 5th residual block, the input end of the 7th residual block receives the result of adding the features output by the output end of the 6th residual block to the features input to the input end of the 6th residual block, the input end of the 8th residual block receives the result of adding the features output by the output end of the 7th residual block to the features input to the input end of the 7th residual block, the input end of the fully connected layer receives the result of adding the features output by the output end of the 8th residual block to the features input to the input end of the 8th residual block, the input end of the Softmax activation layer receives the output value output by the output end of the fully connected layer, and the output end of the Softmax activation layer outputs a true classification probability and a false classification probability as the output end of the discriminator; among them, the output value of the fully connected layer is 2.

6. The method for anti-forensic analysis of audio recompression based on generative adversarial networks according to claim 5 is characterized in that The structures of the first to eighth residual blocks are the same. They are all composed of the ninth convolutional layer, batch normalization layer, and ReLU activation layer connected in sequence. The input of the ninth convolutional layer is the input of the residual block where it is located, the input of the batch normalization layer receives the features output by the output of the ninth convolutional layer, the input of the ReLU activation layer receives the features output by the output of the batch normalization layer, and the output of the ReLU activation layer is the output of the residual block where it is located. Among them, the convolution kernel size of the ninth convolutional layer in the first residual block is 1×30, the number of convolution kernels is 8, and the stride is 2. The convolution kernel size of the ninth convolutional layer in the second residual block is 1×30, the number of convolution kernels is 16, and the stride is 2. The third The convolution kernel size of the ninth convolution layer in the residual block is 1×30, the number of convolution kernels is 32, and the stride is 2. The convolution kernel size of the ninth convolution layer in the fourth residual block is 1×30, the number of convolution kernels is 64, and the stride is 2. The convolution kernel size of the ninth convolution layer in the fifth residual block is 1×30, the number of convolution kernels is 128, and the stride is 2. The convolution kernel size of the ninth convolution layer in the sixth residual block is 1×30, the number of convolution kernels is 256, and the stride is 2. The convolution kernel size of the ninth convolution layer in the seventh residual block is 1×30, the number of convolution kernels is 512, and the stride is 2. The convolution kernel size of the ninth convolution layer in the first residual block is 1×30, the number of convolution kernels is 1024, and the stride is 2.

7. The method for anti-forensic analysis of audio recompression based on generative adversarial networks according to claim 1 is characterized in that The heavy compression adopts AMR compression.

Images