A carrier audio enhancement method based on sparse perturbation
By generating sparse perturbations using generative adversarial networks and embedding secret information using the least significant bit algorithm, the problem of poor auditory quality and insufficient security in existing carrier audio enhancement methods is solved, achieving carrier audio enhancement with high security and good auditory quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-03
- Publication Date
- 2026-04-07
AI Technical Summary
Existing carrier audio enhancement methods based on adversarial examples struggle to adaptively adjust the strength of adversarial perturbations, resulting in poor audio quality and limited security improvements for carrier audio.
A sparse perturbation is constructed using a generative adversarial network. By calculating the temporal and intensity variation characteristics of the audio content, a sparse perturbation is generated and added to the carrier audio. Secret information is embedded by combining the least significant bit audio steganography algorithm, and the generative adversarial network is used for optimization training.
It improves the auditory quality of the carrier audio and effectively misleads the target steganalysis, thereby increasing the security and detection difficulty of the encrypted audio.
Smart Images
Figure CN116884422B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of audio steganography technology, and in particular to a carrier audio enhancement method based on sparse perturbation. Background Technology
[0002] Audio steganography, a key technology in information hiding within cyberspace security, utilizes the information redundancy of digital audio to embed secret messages into the carrier audio, minimizing the possibility of third parties detecting the embedded information. During communication, audio steganography can conceal the transmission of secret messages, effectively reducing the risk of malicious interception. Therefore, it is widely used in covert communication, intelligence transmission, and other cyberspace security fields. Among these, audio steganography algorithms based on the Least Significant Bit (LSB) are one of the most typical audio steganography techniques due to their high efficiency and good security. Its main idea is to replace the least significant bit of each sample point in a discrete sequence of digital audio samples with the corresponding binary information of the secret message. Depending on the specific implementation, it can be further classified into two types of audio steganography algorithms: LSB replacement and LSB matching.
[0003] Audio steganalysis can be viewed as an attack technique corresponding to audio steganography, and it exhibits an adversarial development trend with audio steganography. Early audio steganalyzers based on handcrafted features relied primarily on manually designed statistical features combined with classifiers to achieve audio steganalysis. In recent years, with the rapid development of deep learning technology, researchers have combined audio steganalysis with deep learning, significantly improving its performance. Deep learning-based audio steganalysis algorithms utilize the powerful hierarchical feature learning capabilities of deep neural networks to extract subtle steganalysis features from digital audio to achieve accurate classification of the carrier audio and the steganalyst audio, where the carrier audio is the audio file without embedded secret information. Specifically, in the paper "Audio steganalysis with convolutional neural network," Chen et al. first introduced Convolutional Neural Networks (CNNs) into the field of audio steganalysis, proposing an end-to-end audio steganalysis framework based on CNNs, ChenNet. In the paper "Audio steganalysis with improved convolutional neural network," Lin et al. proposed LinNet, an audio steganalysis method based on an improved CNN. LinNet incorporates a high-pass filter for parameter transformation in the preprocessing layer and adds truncated linear units to enhance the network's learning ability, further improving the detection performance of low-embedding-rate steganography algorithms compared to ChenNet. In the paper "Deep audio steganalysis in time domain," Lee et al. proposed BSNet, an audio steganalysis network based on bit-plane separation. This method does not use a high-pass filter in the preprocessing layer but instead uses bit-plane separation to directly reveal the modified bitstream carrying the encrypted audio, achieving similarly good detection performance. In conclusion, with the rapid development of deep learning-based audio steganalysis technology, the security of existing least significant bit-based audio steganalysis algorithms is under serious threat.
[0004] To enhance the security of audio steganography algorithms, the carrier audio can be modified to resist steganographic analysis attacks. Among these, adversarial example-based audio carrier enhancement methods are the most representative. These methods add adversarial perturbations using adversarial techniques, misleading deep neural network-based steganalysts into making incorrect judgments. Specifically, in the paper "Audio steganography based on iterative adversarial attacks against convolutional neural networks," Wu et al. first applied adversarial techniques to audio steganography. By iteratively updating the distortion cost function using adversarial perturbations, they improved the security of audio steganography until the attack success rate against the target steganalyst exceeded a threshold, making it difficult for the steganalyst to detect the steganographic audio. In the paper "Iteratively generated adversarial perturbation for audio stego post-processing," Ying et al. also designed a steganography post-processing algorithm based on iterative adversarial optimization using adversarial techniques. This algorithm processes the steganized audio carrier by adding adversarial perturbations to enhance security.
[0005] However, the above methods still have shortcomings:
[0006] 1) Carrier audio enhancement methods based on adversarial examples only focus on the global gradient information of the input sample, making it difficult to adaptively adjust the adversarial perturbation intensity for different audio content, resulting in poor auditory quality of carrier audio.
[0007] 2) The performance of adversarial example-based carrier audio enhancement methods often depends on the chosen adversarial example generation algorithm, making it difficult to achieve adversarial optimization against the target steganalysis and resulting in little improvement in audio steganography security. Summary of the Invention
[0008] The purpose of this invention is to provide a carrier audio enhancement method based on sparse perturbation.
[0009] The technical solution for achieving the objective of this invention is as follows:
[0010] A carrier audio enhancement method based on sparse perturbation includes:
[0011] Step 1: Using the training set, perform multiple adversarial training sessions on the discriminator and the adversarial perturbation generator to obtain a trained adversarial perturbation generator.
[0012] The adversarial training specifically includes:
[0013] 1.1 Input the carrier audio c into the adversarial perturbation generator G to obtain the adversarial perturbation v; where the carrier audio c is represented as c = [c1, c2, ..., c N ], N represents the total number of sampling points of the carrier audio, c i This represents the audio intensity at the i-th sampling point;
[0014] 1.2 Calculate the intensity descriptor d = [d1, d2, ..., dc] for the change in carrier audio c. N ]; where d1=d N =0; when i∈{2,3,…,N-1}, In the formula, This represents the relative change intensity of the i-th sampling point in the time dimension. This represents the relative intensity of the i-th sampling point with respect to the carrier audio c. This represents the average intensity of the carrier audio c, i.e. ∈ represents a predefined minimum number;
[0015] 1.3 Input the carrier audio c into K different steganalyzers D t ={D t,1 D t,2 ,…,D t,K}, K≥1; Calculate the carrier audio c for the k-th steganalysis D t,k gradient weight sequence Among them, L t,k This represents the k-th steganalysis unit D after inputting the audio carrier c. t,k The loss function value output during the forward propagation process; calculate the fused gradient weight sequence M. f ,
[0016] 1.4 Calculate the fused gradient weight sequence M f The dot product t between the intensity of change descriptor d and the change dot product t i =M f (i)×d i ; will t i With threshold T m Comparison, that is The binary weight sequence B = [B1, B2, ..., B] is obtained. N ];
[0017] 1.5 Perform a dot product operation between the adversarial perturbation v and the binary weight sequence B to obtain the sparse perturbation v. l ;
[0018] 1.6 The sparse perturbation v l Adding this to the carrier audio c results in the enhanced carrier audio c′, i.e., c′ = v l+c;
[0019] 1.7 Using an audio steganography algorithm based on the least significant bit, the secret information is embedded into the enhanced carrier audio c′ to obtain the enhanced carrier audio s′;
[0020] 1.8 Based on the discriminator loss function L GAN_D The discriminator D is updated using the Adam optimization method to obtain the updated discriminator D′;
[0021] The discriminator loss function L GAN_D =(D(s′)-y s ) 2 +(D(c+B⊙v)-y c ) 2 ;
[0022] Where D(s′) represents the probability that the enhanced encrypted audio s′ is classified as encrypted audio by the discriminator D, and y s The tag indicating encrypted audio, y s =1; D(c+B⊙v) represents the probability that the carrier audio c is classified as encrypted audio by the discriminator D, y c The tag representing the carrier audio, y c =0; ⊙ represents the dot product operation;
[0023] 1.9 Based on the overall loss function L of the adversarial perturbation generator G G The Adam optimization method is used to update the adversarial perturbation generator.
[0024] G, thus obtaining the updated adversarial perturbation generator G′;
[0025] The overall loss function L of the adversarial perturbation generator G G =αL GAN_G +βL adv Where α represents the loss L against the perturbation generator. GAN_G The weight, β, represents the adversarial loss L. adv The weights;
[0026] The anti-disturbance generator loss L GAN_G =(D′(s′+B⊙G(c))-y c ) 2 In the formula, D′(s′+B⊙G(c)) represents the probability that the enhanced encrypted audio s′ is classified as encrypted audio by the discriminator D′ updated in step 1.8, and G(c) represents the adversarial perturbation generated by the adversarial perturbation generator G;
[0027] The resistance loss L adv The process is as follows: The enhanced, encrypted audio s′ is input into the pre-trained steganalyzer D. tThe classification loss of the steganalysis is used as the adversarial loss L. adv ,
[0028] Right now
[0029] Step 2: Using the trained adversarial perturbation generator, the carrier audio to be enhanced is processed in the same way as in steps 1.1-1.5 to obtain sparse perturbations.
[0030] Step 3: Add sparse perturbations to the carrier audio to be enhanced to obtain the enhanced carrier audio.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] 1. Generative adversarial networks are used to construct adversarial perturbations. Sparse perturbations are calculated using the temporal and intensity variation characteristics of audio content to achieve carrier audio enhancement and improve the auditory quality of carrier audio.
[0033] 2. Compared with existing carrier audio enhancement algorithms, it has higher security. That is, the enhanced carrier audio obtained by this invention can effectively mislead the target steganalysis after embedding secret information, causing it to produce incorrect detection results. Attached Figure Description
[0034] Figure 1 This is an overall framework diagram of the present invention.
[0035] Figure 2 This is a network structure diagram of the anti-disturbance generator in a specific embodiment of the present invention.
[0036] Figure 3 This is a network structure diagram of the discriminator in a specific embodiment of the present invention. Detailed Implementation
[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0038] An audio enhancement method based on sparse perturbation using generative adversarial networks (GANs) is proposed. This method constructs adversarial perturbations using GANs and calculates sparse perturbations based on the temporal and intensity variation characteristics of the audio content. These sparse perturbations are then added to the carrier audio to obtain enhanced carrier audio. The resulting encrypted audio, after embedding secret information, can effectively mislead target steganalyzers, thus improving the security of the encrypted audio.
[0039] A. Training phase (e.g.) Figure 1 As shown):
[0040] Step 1. Select a carrier audio c from the training set and input it into the adversarial perturbation generator G. The adversarial perturbation generator G generates an adversarial perturbation v based on the carrier audio. The carrier audio is represented as c = [c1, c2, ..., c...]. N ], N represents the total number of sampling points of the carrier audio, c i This represents the audio intensity at the i-th sampling point. In this example, the network structure of the adversarial perturbation generator G is as follows: Figure 2 As shown. While maintaining the same length for both the input and output sequences, other network structures can also be used for the adversarial perturbation generator G.
[0041] Step 2. First, calculate the intensity descriptor d = [d1, d2, ..., d] of the input carrier audio. N When the sampling point number i∈{2,3,…,N-1}, the calculation formula is as follows:
[0042]
[0043] in, This represents the relative intensity of change of the i-th audio sample point in the time dimension. This represents the relative intensity of the i-th audio sample point and the input carrier audio signal. in, This represents the average intensity of the input carrier audio, i.e. ∈ represents a very small number to prevent the denominator from being 0. In this example, the value ∈ = 10. -5 When the sampling point number i = 1 or N, the change intensity descriptor d corresponding to that sampling point... i =0.
[0044] Step 3. Input the carrier audio c into K different pre-trained steganalysts D. t ={D t,1 D t,2 ,…,D t,K}, where K represents the total number of steganalysis analyzers; in this example, K = 3. Then, calculate the carrier audio c for the k-th steganalysis analyzer D. t,k gradient weight sequence Where L t,k This represents the k-th steganalysis unit D after inputting the audio carrier c. t,k The loss function value output during the forward propagation process. Next, the fused gradient weight sequence M is calculated. f The calculation formula is as follows:
[0045]
[0046] Step 4. Calculate the fused gradient weight sequence M fThe dot product between the change intensity descriptor d and the preset threshold T m The comparison is performed to achieve binarization, resulting in a binary weight sequence B = [B1, B2, ..., B...]. N Threshold T m Set it to 0.3. The above process can be expressed as the following formula:
[0047] t i =M f (i)×d i
[0048]
[0049] Step 5. Perform a dot product operation between the adversarial perturbation v and the binary weight sequence B to obtain the sparse perturbation v. l .
[0050] Step 6. Transfer the sparse perturbation v l Adding this to the carrier audio c results in the enhanced carrier audio c′, i.e., c′ = v l +c.
[0051] Step 7. Use the Least Significant Bit Replacement (LSBR) algorithm or the Least Significant Bit Matching (LSBM) algorithm as the least significant bit-based audio steganography algorithm in this example to embed the secret information into the enhanced carrier audio c′, resulting in the enhanced steganography audio s′.
[0052] Step 8. Input the enhanced carrier audio s′ and the carrier audio c into the discriminator D. The discriminator D has a one-dimensional convolutional neural network structure, and the output layer of the discriminator has 2 neurons, as shown below. Figure 3 As shown. While ensuring the output layer has 2 neurons, the discriminator D can also use other network structures. Construct the discriminator loss function L. GAN_D , used to optimize the discriminator D, and used for adversarial training with the adversarial perturbation generator G, is calculated as follows:
[0053] L GAN_D =(D(s′)-y s ) 2 +(D(c+B⊙v)-y c ) 2
[0054] Where D(s′) represents the probability that the enhanced encrypted audio s′ is classified as encrypted audio by the discriminator D, and y s The tag indicating encrypted audio, y s=1; D(c+B⊙v) represents the probability that the carrier audio c is classified as encrypted audio by the discriminator D, y c The tag representing the carrier audio, y c =0; ⊙ represents the dot product operation.
[0055] Based on the discriminator loss function L GAN_D The network parameters of the discriminator D are updated using the Adam optimization method to obtain the updated discriminator D′. During this process, the network parameters of the adversarial perturbation generator G remain fixed and are not updated.
[0056] Step 9. Input the carrier audio c into the adversarial perturbation generator G. The overall loss function L of the adversarial perturbation generator. G Loss L from the adversarial perturbation generator GAN_G and combat losses L adv The weighted summation is obtained, and the calculation formula is as follows:
[0057] L G =αL GAN_G +βL adv
[0058] Where α represents the loss of the adversarial perturbation generator L GAN_G The weight, β, represents the adversarial loss L. adv The weights. In this example, the adversarial perturbation generator loss L... GAN_G The weight α and the adversarial loss L adv The weights β are all set to 1.0.
[0059] Among them, the loss of the anti-disturbance generator L GAN_G The least squares loss method is used for calculation, and the formula is as follows:
[0060] L GAN_G =(D′(s′+B⊙G(c))-y c ) 2
[0061] The enhanced encrypted audio s′ is input into the pre-trained steganalyzer D. t The classification loss of the steganalysis is used as the adversarial loss L. adv The calculation formula is as follows:
[0062]
[0063]
[0064] Among them, D t,k This represents the k-th pre-trained steganalysis.
[0065] According to L GThe network parameters of the adversarial perturbation generator G are updated using the Adam optimization method, while the network parameters of the discriminator D′ remain fixed and are not updated during this process.
[0066] Step 10. During the training phase, repeat steps 1 through 9 a total of 20,000 times, i.e., T. max =20,000, complete adversarial training, and obtain the trained adversarial perturbation generator G. * .
[0067] B. Application Phase:
[0068] Step 1. Input the carrier audio c to be enhanced from the test set into the trained adversarial perturbation generator G. * And perform steps 1 to 5 of the training phase to obtain the sparse perturbation v. l .
[0069] Step 2. Transfer the sparse perturbation v l Add it to the carrier audio c to be enhanced to obtain the enhanced carrier audio c′.
[0070] To illustrate the performance of the above-described solution of the present invention, relevant experiments were also conducted.
[0071] Experimental parameters: The dataset used is the TIMIT speech corpus, widely used in the field of audio steganography. The dataset processing method is as follows: 10 audio blocks of 0.1 seconds each are randomly extracted from each audio file, resulting in 10 × 62,980 = 629,800 audio blocks. The dataset is randomly divided into training and testing sets in a 6:4 ratio. The training set is used to train the adversarial perturbation generator, while the testing set is used to evaluate the security of the enhanced adversarial audio. LSB replacement (LSBR) and LSB matching (LSBM) algorithms are used as the least significant bit-based audio steganography algorithms in the experiment, with an embedding rate of 1.0 bps. During the training phase, pre-trained steganalysts ChenNet, LinNet, and BSNet are used. ChenNet is from the paper "Audio steganalysis with convolutional neural network", LinNet is from the paper "Audiosteganalysis with improved convolutional neural network", and BSNet is from the paper "Deepaudio steganalysis in time domain". The training method is the same as in the original papers. The test steganalysis used in the application phase is the same as the pre-trained steganalysis used in the training phase. The adversarial perturbation generator loss L... GAN_G The weight α and the adversarial loss L advThe weights β are all set to 1.0. The threshold T of the binary weighted sequence B... m Set the value to 0.3. During training, set the batch size to 64, select the Adam optimizer, and set the learning rate to 10. -4 The experiment primarily tests the security of this invention, specifically using a target steganalyzer encountered during the training phase for detection. The experimental metric is the false negative rate, which is the percentage of encrypted audio that is misidentified as carrier audio by the steganalyzer. The experiment will compare the invention with existing carrier audio enhancement algorithms to verify its effectiveness.
[0072] Table 1. Security Tests Under the Target Steganalysis Detection of the Invention.
[0073]
[0074] Table 1 shows the experimental results for embedding rates of 1.0 bps in Embodiment 1 of this invention, with the best results indicated in bold. FGSM utilizes the FGSM algorithm from the paper "Explaining and harnessing adversarial examples" to quickly generate adversarial perturbations, which are then added to the carrier audio to achieve carrier enhancement. IA-SPP refers to a comparison with the adversarial audio constructed using the iteratively generated adversarial perturbation algorithm proposed by Ying et al. in the paper "Iteratively generated adversarial perturbation for audiostego post-processing". Experimental results show that the adversarial perturbation generators obtained from different steganalysts exhibit varying security performance. However, overall, the method of this invention demonstrates superior security performance under target steganalyst detection compared to other carrier audio enhancement algorithms. Furthermore, the peak signal-to-noise ratio (PSNR) of the carrier audio before and after enhancement was calculated for different enhancement algorithms. For the samples in the test set, the average PSNR of this invention was 36.05 dB, higher than the 30.21 dB of the FGSM algorithm and the 32.76 dB of the IA-SPP algorithm. The above results demonstrate that the algorithm of this invention can maintain good auditory quality in the enhanced carrier audio.
Claims
1. A carrier audio enhancement method based on sparse perturbation, characterized in that, include: Step 1: Using the training set, perform multiple adversarial training sessions on the discriminator and the adversarial perturbation generator to obtain a trained adversarial perturbation generator. The adversarial training specifically includes: 1.1 Input the carrier audio c into the adversarial perturbation generator G to obtain the adversarial perturbation v; where the carrier audio c is represented as c = [c1, c2, ..., c2]. N ], N represents the total number of sampling points of the carrier audio, c i This represents the audio intensity at the i-th sampling point; 1.2 Calculate the intensity descriptor d = [d1, d2, ..., dc] for the change in carrier audio c. N ]; where d1=d N =0; when i∈{2,3,...,N-1}, In the formula, This represents the relative change intensity of the i-th sampling point in the time dimension. This represents the relative intensity of the i-th sampling point with respect to the carrier audio c. This represents the average intensity of the carrier audio c, i.e. ∈ represents a predefined minimum number; 1.3 Input the carrier audio c into K different steganalyzers D t ={D t，1 D t，2 , ..., D t，K }, K≥1; The calculation of the carrier audio c for the k-th steganalysis D t，k gradient weight sequence Among them, L t，k This represents the k-th steganalysis unit D after inputting the audio carrier c. t，k The loss function value output during the forward propagation process; Calculate the fused gradient weight sequence M f , 1.4 Calculate the fused gradient weight sequence M f The dot product t between the intensity of change descriptor d and the change dot product t i =M f (i)×d i ; will t i With threshold T m Comparison, that is The binary weight sequence B = [B1, B2, ..., B] is obtained. N ]; 1.5 Perform a dot product operation between the adversarial perturbation v and the binary weight sequence B to obtain the sparse perturbation v. l ; 1.6 The sparse perturbation v l Adding this to the carrier audio c results in the enhanced carrier audio c′, i.e., c′ = v l +c; 1.7 Using an audio steganography algorithm based on the least significant bit, the secret information is embedded into the enhanced carrier audio c′ to obtain the enhanced carrier audio s′; 1.8 Based on the discriminator loss function L GAN_D The discriminator D is updated using the Adam optimization method to obtain the updated discriminator D′; The discriminator loss function L GAN_D =(D(s′)-y s ) 2 +(D(c+B⊙v)-y c ) 2 ; Where D(s′) represents the probability that the enhanced encrypted audio s′ is classified as encrypted audio by the discriminator D, and y s The tag indicating encrypted audio, y s =1; D(c+B⊙v) represents the probability that the carrier audio c is classified as encrypted audio by the discriminator D, y c The tag representing the carrier audio, y c =0; ⊙ represents the dot product operation; 1.9 Based on the overall loss function L of the adversarial perturbation generator G G The Adam optimization method is used to update the adversarial perturbation generator G, resulting in the updated adversarial perturbation generator G′. The overall loss function L of the adversarial perturbation generator G G =αL GAN_G +βL adv Where α represents the loss L against the perturbation generator. GAN_G The weight, β, represents the adversarial loss L. adv The weights; The anti-disturbance generator loss L GAN_G =(D′(s′+B⊙G(c))-y c ) 2 In the formula, D′(s′+B⊙G(c)) represents the probability that the enhanced encrypted audio s′ is classified as encrypted audio by the discriminator D′ updated in step 1.8, and G(c) represents the adversarial perturbation generated by the adversarial perturbation generator G; The resistance loss L adv The process is as follows: The enhanced, encrypted audio s′ is input into the pre-trained steganalyzer D. t The classification loss of the steganalysis analyzer is used as the adversarial loss L. adv , Right now Step 2: Using the trained adversarial perturbation generator, the carrier audio to be enhanced is processed in the same way as in steps 1.1-1.5 to obtain sparse perturbations. Step 3: Add sparse perturbations to the carrier audio to be enhanced to obtain the enhanced carrier audio.