Audio robust watermark generation method based on embedding strength prediction network and terminal

Abstract

The invention discloses an audio robust watermark generation method based on an embedding strength prediction network and a terminal, and the method comprises the steps: obtaining audio data, carrying out the framing and watermark embedding processing of the audio data, and generating a corresponding three-dimensional coefficient matrix; inputting the three-dimensional coefficient matrix into an embedding strength prediction network, and outputting a corresponding embedding strength factor; adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame embedded with watermark information; the invention provides a new self-adaptive audio robust watermark generation method based on an embedded strength prediction network, and the imperceptibility and robustness of a watermark algorithm are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of audio robust watermarking, and in particular to an audio robust watermarking generation method and a terminal based on an embedded strength prediction network. Background Art

[0002] The rapid development of the Internet has led to the widespread dissemination of digital media, among which audio has played an important role in daily life and social interaction due to its convenient communication method. The popularity of audio editing software and portable recording devices has simplified the production and sharing of audio, but it has also brought about copyright and privacy security issues. Ensuring the authenticity and copyright protection of audio has become an urgent problem that needs to be solved. In recent years, the emerging technology of deep learning has been widely used in traditional digital robust watermarking technology for copyright protection, which has attracted widespread attention. Audio robust watermarking is a technology that embeds and extracts corresponding information in audio signals, aiming to protect the integrity and copyright of audio content. Audio robust watermarking algorithms based on deep learning have become an important research hotspot, and in-depth research on them is of great significance.

[0003] However, existing deep learning-based audio watermarking methods generally lack consideration of differences in embedding strength when processing different audios. These algorithms are usually trained on large datasets and use the same embedding strength for all audios in practical applications. However, for a single audio, a fixed embedding strength makes it difficult to optimize both transparency and robustness. Each audio has different characteristics. When embedding a watermark, a fixed strength may cause the watermark to be too obvious in some audios, affecting the auditory experience, or not obvious enough in some audios, reducing robustness.

[0004] Therefore, the prior art needs to be improved. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides an audio robust watermark generation method, terminal and storage medium based on an embedding strength prediction network to solve the technical problem that the traditional deep learning-based audio watermark algorithm lacks dynamic adjustment of the embedding strength.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides an audio robust watermark generation method based on an embedded strength prediction network, comprising:

[0008] Acquire audio data, perform frame segmentation and watermark embedding processing on the audio data, and generate a corresponding three-dimensional coefficient matrix;

[0009] Inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor;

[0010] The embedding strength of the watermark is adjusted according to the embedding strength factor, and an audio frame embedded with the watermark information is output.

[0011] In one implementation, the acquiring of audio data, performing frame segmentation and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix includes:

[0012] Acquire the audio data, perform frame processing on the audio data, and perform short-time Fourier transform processing on each frame of audio after the frame division to obtain a corresponding two-dimensional coefficient matrix;

[0013] The real part and the imaginary part of the two-dimensional coefficient matrix are concatenated and watermark embedded to obtain the three-dimensional coefficient matrix.

[0014] In one implementation, the short-time Fourier transform processing is performed on each frame of audio after framing to obtain a corresponding two-dimensional coefficient matrix, including:

[0015] Weighting is performed by multiplying the sample points of each frame of audio after framing by the corresponding value of the Hanning window function;

[0016] The corresponding two-dimensional coefficient matrix is ​​obtained by short-time Fourier transform for each frame of audio after windowing.

[0017] In one implementation, inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor includes:

[0018] Input the three-dimensional coefficient matrix into the embedding strength prediction network, and obtain the embedding strength factor corresponding to each frame of audio through feature concatenation and mapping processing;

[0019] The embedding strength prediction network outputs the embedding strength factor corresponding to each frame of audio.

[0020] In one implementation, the step of inputting the three-dimensional coefficient matrix into the embedding strength prediction network and obtaining the embedding strength factor corresponding to each frame of audio through feature concatenation and mapping processing includes:

[0021] Inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix;

[0022] Concatenate the feature representation matrix and the two-dimensional coefficient matrix on the channel and map them back to the initial number of channels;

[0023] The embedding strength factor is determined according to the mapped feature representation matrix.

[0024] In one implementation, the inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix includes:

[0025] The three-dimensional coefficient matrix is ​​sequentially input into a convolution module and four channel attention compression and excitation modules for feature enhancement to obtain a corresponding feature representation matrix.

[0026] In one implementation, concatenating the feature representation matrix with the two-dimensional coefficient matrix on the channel and mapping back to the initial number of channels includes:

[0027] The feature representation matrix and the two-dimensional coefficient matrix are concatenated on the channel, and mapped back to the initial number of channels through a convolution module and a pooling module;

[0028] The output range of the feature representation matrix is ​​mapped to the 0-1 interval through the sigmoid activation function.

[0029] In one implementation, adjusting the embedding strength of the watermark according to the embedding strength factor and outputting the audio frame embedded with the watermark information includes:

[0030] adjusting the embedding strength of the watermark according to the embedding strength factor, and embedding watermark information into the audio frame according to the embedding strength of the watermark;

[0031] The audio frame with the embedded watermark information is output.

[0032] In a second aspect, the present invention also provides a terminal, comprising: a processor and a memory, wherein the memory stores an audio robust watermark generation program based on an embedded strength prediction network, and when the audio robust watermark generation program based on an embedded strength prediction network is executed by the processor, it is used to implement the operation of the audio robust watermark generation method based on an embedded strength prediction network as described in the first aspect.

[0033] In a third aspect, the present invention also provides a computer-readable storage medium, which stores an audio robust watermark generation program based on an embedded strength prediction network. When the audio robust watermark generation program based on an embedded strength prediction network is executed by a processor, it is used to implement the operation of the audio robust watermark generation method based on an embedded strength prediction network as described in the first aspect.

[0034] The present invention adopts the above technical solution to achieve the following effects:

[0035] The present invention can generate a corresponding three-dimensional coefficient matrix by acquiring audio data, performing frame division and watermark embedding processing on the audio data; then, the three-dimensional coefficient matrix is ​​input into an embedding strength prediction network, and a corresponding embedding strength factor is output; finally, the embedding strength of the watermark is adjusted according to the embedding strength factor, and an audio frame with embedded watermark information is output, so that the embedding strength is dynamically adjusted according to the characteristics of each audio, which can not only better protect the copyright, but also improve the anti-attack ability of the watermark without affecting the sound quality; the present invention proposes a new method for generating adaptive audio robust watermarks based on an embedding strength prediction network, which improves the imperceptibility and robustness of the watermark algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of an audio robust watermark generation method based on an embedded strength prediction network in one implementation of the present invention.

[0038] Figure 2 It is a structural schematic diagram of an adaptive audio robust watermark generation network based on an embedded strength prediction network in one implementation of the present invention.

[0039] Figure 3 It is a schematic diagram of the structure of an embedded intensity prediction network in one implementation of the present invention.

[0040] Figure 4 It is a schematic diagram of embedding strength factors of different audios in one implementation of the present invention.

[0041] Figure 5 It is a schematic diagram of embedding strength prediction for different data sets in one implementation of the present invention.

[0042] Figure 6 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] The present invention provides an audio robust watermark generation method, terminal and storage medium based on an embedded strength prediction network. In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] Exemplary Methods

[0046] Existing deep learning-based audio watermarking methods generally lack consideration of differences in embedding strength when processing different audios. These algorithms are usually trained on large datasets and use the same embedding strength for all audios in practical applications. However, for a single audio, a fixed embedding strength makes it difficult to optimize both transparency and robustness. The characteristics of each audio are different. When embedding a watermark, a fixed strength may cause the watermark to be too obvious in some audios, affecting the auditory experience, or not obvious enough in some audios, reducing robustness. Therefore, in order to achieve optimal transparency and robustness, it is necessary to dynamically adjust the embedding strength according to the characteristics of each audio. This dynamic adjustment can not only better protect copyright, but also improve the watermark's anti-attack ability without affecting the sound quality.

[0047] Although deep learning-based audio watermarking technology shows great potential in copyright protection, its embedding strength optimization problem in practical applications still needs further research and improvement. By dynamically adjusting the embedding strength at the level of a single audio, it is possible to better balance transparency and robustness and promote the development of this field.

[0048] In view of the above technical problems, an embodiment of the present invention provides an audio robust watermark generation method based on an embedded strength prediction network, aiming to propose a new adaptive audio robust watermark generation method based on an embedded strength prediction network to improve the imperceptibility and robustness of the watermark algorithm.

[0049] like Figure 1 As shown, an embodiment of the present invention provides an audio robust watermark generation method based on an embedded strength prediction network, comprising the following steps:

[0050] Step S100, obtaining audio data, performing frame segmentation and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix.

[0051] In this embodiment, the audio robust watermark generation method based on the embedded strength prediction network is applied to a terminal, and the terminal includes but is not limited to: computers and mobile terminals and other devices; the terminal is provided with a training migration platform for the audio robust watermark generation model based on the embedded strength prediction network.

[0052] In this embodiment, an adaptive audio robust watermark generation method based on an embedding strength prediction network is proposed. By introducing an embedding strength prediction network to select appropriate embedding strengths for different audio frames, the imperceptibility and robustness of the watermark algorithm are further improved, thereby outputting appropriate embedding strengths for different audio frames to improve the performance of the algorithm. This dynamic adjustment can not only better protect copyright, but also improve the watermark's anti-attack capability without affecting the sound quality.

[0053] like Figure 2 As shown in the figure, this application adds an embedding strength prediction network based on the codec, which acts between the encoder and the distortion layer, takes each audio frame as input, outputs an embedding strength factor, and improves imperceptibility and robustness by selecting appropriate parameters for each audio frame. The specific structure of the network is shown in Figure 2 As shown, the overall framework of the network contains an encoder, a distortion layer, a decoder, and an embedding strength prediction network.

[0054] In the research and application of audio watermarking technology, embedding strength plays a vital role. In essence, the embedding strength prediction network is more like an auxiliary network structure, which is characterized by being independent of the original backbone network (i.e., encoder-distortion layer-decoder) and can be used as a plug-in. The main responsibility of the embedding strength prediction network is to provide the encoder with an optimal embedding strength factor selection based on the content of the audio itself and the watermark information to be embedded. The so-called "optimal" strength factor means that under the adjustment of this factor, the decoder can accurately parse the secret information hidden in the watermark, while also being able to reduce the strength required in the watermark embedding process as much as possible, thereby maximizing the potential of the audio without increasing the bit error rate and achieving a high degree of concealment of the watermark. In other words, the original intention of the design of the embedding strength prediction network is to fully explore and utilize the characteristics of the audio itself without sacrificing audio quality and increasing the bit error rate, so as to achieve the dual purpose of effectively hiding the watermark information and maintaining the audio auditory quality.

[0055] In this embodiment, first, the audio data is read from the original audio file. In order to facilitate processing and analysis, the audio data will be divided into a series of non-overlapping short-time frames, and a short-time Fourier transform is applied to each audio frame to obtain a three-dimensional real-valued STFT coefficient matrix. This three-dimensional coefficient matrix contains the frequency domain information of the audio and is the basis for watermark embedding and extraction.

[0056] Specifically, in an implementation of this embodiment, step S100 includes the following steps:

[0057] Step S110, acquiring the audio data, performing frame processing on the audio data, and performing short-time Fourier transform processing on each frame of audio after framing to obtain a corresponding two-dimensional coefficient matrix;

[0058] Step S120, concatenating the real part and the imaginary part of the two-dimensional coefficient matrix, and performing watermark embedding processing to obtain the three-dimensional coefficient matrix.

[0059] In this embodiment, audio data is first read from the original audio file. Audio clips can be randomly selected from data sets such as LibriSpeech, FMA, LJSpeech, and VCTK, and the length of these clips is set to 161,290, that is, a total of ten audio frames. For the input audio file, this application adopts a frame-by-frame embedding method. First, the audio is divided into non-overlapping frames with a fixed length for subsequent watermark embedding and extraction. For a length of L A Carrier audio A={A(a)|1≤a≤L A}, A(a) represents the a-th sample point, with frame length L F (L F =16129) Divide A into n non-overlapping frames: A1, A2, ..., A n , for audio end sample points less than L F The frames are not watermarked.

[0060] In this embodiment, each frame of audio A after frame division i (i=1,…,n), and make a window with a length of N and a window shift of L h The short-time Fourier transform of Represents the size of the matrix after STFT transformation. One dimension represents the number of time frames and the second dimension represents the number of frequency groups.

[0061] In this embodiment, the two-dimensional coefficient matrix S i The real and imaginary parts of are concatenated on the channel, and the concatenated matrix is ​​watermarked to obtain a three-dimensional coefficient matrix It is used as the input of the subsequent encoder and embedding strength prediction network. The purpose of this is to facilitate the subsequent deep learning model processing, that is, to convert complex data into a real data format that is easy for the neural network to process.

[0062] Specifically, in an implementation of this embodiment, step S110 includes the following steps:

[0063] Step S111, weighting the sample points of each frame of audio after framing by the corresponding value of the Hanning window function;

[0064] Step S112, performing short-time Fourier transform on each frame of windowed audio to obtain a corresponding two-dimensional coefficient matrix.

[0065] In this embodiment, the process of adding a window function is an important step in digital signal processing, especially when performing a short-time Fourier transform, the audio signal is divided into a series of non-overlapping or partially overlapping short-time windows, and then a fast Fourier transform (FFT) is applied to each window to obtain the spectrum information within the time period. Windowing can reduce boundary effects and smooth frame edges.

[0066] In this embodiment, for each frame of audio A after framing, i (i=1,…,n), and make a window with a length of N and a window shift of L h The short-time Fourier transform of the corresponding two-dimensional complex matrix S i :

[0067]

[0068] in, Indicates the index value of the window, represents the frequency index, and g(c) represents the Hanning window function:

[0069]

[0070] In this embodiment, the sample points of each frame of audio after framing are multiplied by the corresponding value of the Hanning window function to be weighted. The windowing step occurs after the audio signal is divided into frames and before the STFT is performed, ensuring that the processing of each audio frame can effectively reduce the edge effect and obtain more accurate spectrum analysis results, which is crucial for subsequent audio robust watermark embedding.

[0071] In this embodiment, the audio signal is first divided into non-overlapping frames, which is helpful for subsequent frame-by-frame processing; and watermark embedding is not performed on frames with insufficient audio end samples to avoid edge effects; each frame of audio is represented by a short-time Fourier transform (STFT), and the spectrum is smoothed using a Hanning window function to ensure the accuracy of the transformation.

[0072] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for generating an audio robust watermark based on an embedded strength prediction network further includes the following steps:

[0073] Step S200: input the three-dimensional coefficient matrix into an embedding strength prediction network, and output a corresponding embedding strength factor.

[0074] In this embodiment, the structure of the embedding strength prediction network is as follows Figure 3As shown in the figure, the three-dimensional short-time Fourier transform coefficient matrix after embedding the watermark As input, an embedding strength factor α is output. The embedding strength prediction network provides a customized embedding strength factor for each frame through deep analysis of the audio frame to optimize the watermark embedding process and ensure the best performance balance under different audio content and conditions.

[0075] Specifically, in an implementation of this embodiment, step S200 includes the following steps:

[0076] Step S210, inputting the three-dimensional coefficient matrix into the embedding strength prediction network, and obtaining the embedding strength factor corresponding to each frame of audio through feature concatenation and mapping processing;

[0077] Step S220: outputting an embedding strength factor corresponding to each frame of audio through the embedding strength prediction network.

[0078] In this embodiment, the embedding strength prediction network includes multiple feature extraction modules, which perform feature extraction on the input three-dimensional coefficient matrix to obtain a higher-level feature representation. Through multiple such modules, the network can capture the features of the audio frame while keeping the size of the feature map unchanged. The extracted feature representation is spliced ​​with the original three-dimensional coefficient matrix on the channel to form a feature matrix containing more information. The spliced ​​feature matrix is ​​mapped back to the initial number of channels through network layers such as convolutional layers, ensuring efficient integration and transmission of information. After mapping, the embedding strength prediction network outputs a value between 0 and 1, representing the embedding strength factor of the audio frame. This embedding strength factor will be used to adjust the strength of the watermark to achieve a balance between imperceptibility and robustness.

[0079] Specifically, in an implementation of this embodiment, step S210 includes the following steps:

[0080] Step S211, inputting the three-dimensional coefficient matrix into multiple feature extraction modules to obtain a corresponding feature representation matrix;

[0081] Step S212, concatenating the feature representation matrix and the two-dimensional coefficient matrix on the channel, and mapping them back to the initial number of channels;

[0082] Step S213: determining the embedding strength factor according to the mapped feature representation matrix.

[0083] In this embodiment, in the feature extraction part, the channel attention-based compression and excitation block (SE block) is selected as the basic module. The SE module can effectively capture the importance of features and recalibrate weights to improve the network's attention to key information. Through 4 feature extraction modules, the corresponding feature representation is obtained while keeping the shape and size unchanged.

[0084]

[0085] In this embodiment, the feature representation With the two-dimensional coefficient matrix S i After concatenation on the channel, the convolution layer is used to map the initial number of channels, and the sigmoid activation function is used to map the output range to the 0-1 interval:

[0086]

[0087] In this embodiment, in order to ensure the stability of network training, the output embedding strength factor α may be controlled within the range of 0.7 to 1.3, ensuring that the watermark has sufficient robustness without compromising the audio quality.

[0088]

[0089] In this embodiment, the output embedding strength factor is then used to adjust the embedding process of the watermark, and the embedding degree of each audio frame is adjusted according to its content and the predicted strength factor, so as to achieve personalization of the watermark depth.

[0090] Specifically, in an implementation of this embodiment, step S211 includes the following steps:

[0091] Step S211a, input the three-dimensional coefficient matrix into a convolution module and four channel attention compression and excitation modules in sequence for feature enhancement to obtain a corresponding feature representation matrix.

[0092] In this embodiment, the constructed three-dimensional coefficient matrix is ​​fed into a network structure consisting of a convolution module and four compression and excitation modules based on a channel attention mechanism. First, the network uses one or more convolution layers to process the three-dimensional coefficient matrix for preliminary feature extraction. The convolution layer can capture local spatiotemporal features and provide a basis for subsequent feature enhancement. Then, these features are further enhanced through four consecutive SE modules. The core function of the SE module is to "squeeze" the feature map through global average pooling, reduce the dimension to the channel dimension, and then "excite" through two fully connected layers to learn the importance weight of each channel. These weights are used to recalibrate the previous features, emphasize the channels that are more important to the task, and thus achieve selective enhancement of features. After being processed by the above-mentioned convolution module and channel attention module, the features of each audio frame are significantly enhanced, forming a more refined and discriminative feature representation matrix. This matrix not only retains the key information of the audio frame, but also highlights the parts that are conducive to predicting the embedding strength.

[0093] Specifically, in an implementation of this embodiment, step S212 includes the following steps:

[0094] Step S212a, concatenating the feature representation matrix and the two-dimensional coefficient matrix on the channel, and mapping them back to the initial number of channels through a convolution module and a pooling module;

[0095] Step S212b, mapping the output range of the feature representation matrix to a 0-1 interval through a sigmoid activation function.

[0096] In this embodiment, first, the feature representation matrix obtained by the channel attention compression and excitation module is spliced ​​with the original two-dimensional complex coefficient matrix in the channel dimension. This combines the high-level abstracted feature information with the frequency domain representation of the original audio, allowing the model to better understand the characteristics of each audio frame and make more accurate intensity predictions based on this. The spliced ​​matrix is ​​then fed into a convolution module, which may contain multiple convolution layers to further extract and integrate key information from the spliced ​​features. The convolution module helps capture local features and patterns, while the pooling module is used to reduce the spatial dimension of the feature map while maintaining the most important feature information, thereby reducing computational complexity and enhancing robustness to spatial changes. This process ultimately maps the features back to the initial number of channels to ensure compatibility with subsequent processing.

[0097] In this embodiment, the features obtained by the above processing are convolved and pooled, and then passed through a sigmoid activation function. The sigmoid function is an S-shaped function whose output range is strictly limited to (0,1), which is suitable for probability estimation or scenarios where the output value needs to be constrained to a specific interval. In this application, it is used to map the output of the feature representation matrix to the interval from 0 to 1, and obtain the embedding strength factor for each audio frame. These factors directly indicate the strength that should be used when embedding the watermark in each frame to achieve the best balance between transparency and robustness. Larger output values ​​(close to 1) indicate higher embedding strength, which means that the watermark is more robust but may have a greater impact on audio quality; conversely, smaller values ​​(close to 0) indicate weaker embedding, which improves imperceptibility but may sacrifice some robustness.

[0098] In this embodiment, the embedding strength prediction network is used to dynamically adjust the embedding strength of the audio watermark to improve imperceptibility and robustness. The three-dimensional coefficient matrix is ​​input into the embedding strength prediction network, and features are extracted through multiple feature extraction modules and channel attention mechanisms. It is spliced ​​with the two-dimensional coefficient matrix and mapped back to the initial number of channels, and finally the embedding strength factor in the range of 0-1 is output through the sigmoid activation function. In this way, the watermark embedding strength is dynamically adjusted to ensure that the transparency and robustness of the audio robust watermark are optimized.

[0099] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for generating an audio robust watermark based on an embedded strength prediction network further includes the following steps:

[0100] Step S300: adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting the audio frame embedded with the watermark information.

[0101] In this embodiment, by predicting the embedding strength factor for each audio frame separately, the embedding strength of the watermark can be dynamically adjusted according to the characteristics of the audio content. For noisy or complex parts of the audio, the algorithm can predict a smaller embedding strength to ensure the imperceptibility of the watermark, so that the watermark has almost no effect on the listening experience of the original audio. On the contrary, for the stable or strong signal parts of the audio, a larger embedding strength factor can be used to ensure that the watermark can still be accurately extracted even after being attacked, thereby improving the robustness of the watermark. The encoder uses the adjusted embedding strength factor to embed the watermark information into the audio frame. Afterwards, the decoder can try to extract the watermark information in both the original audio and the audio subjected to various attacks to verify the integrity and copyright of the audio.

[0102] Specifically, in an implementation of this embodiment, step S300 includes the following steps:

[0103] Step S310, adjusting the embedding strength of the watermark according to the embedding strength factor, and embedding watermark information into the audio frame according to the embedding strength of the watermark;

[0104] Step S320: output the audio frame embedded with the watermark information.

[0105] In this embodiment, the strength of the watermark signal is adjusted by embedding a strength factor (α) according to the characteristics of each audio frame. This means that the strength of watermark embedding is not fixed, but is adjusted according to the dynamic changes of the audio content, thereby maximizing the concealment and anti-attack capabilities of the watermark without significantly affecting the listening experience.

[0106] In this embodiment, the input embedding strength factor embeds the watermark information into the audio frame through the following formula, and the strength of the watermark is controlled by the embedding strength factor, so as to achieve a balance between imperceptibility and robustness:

[0107]

[0108] Among them, S i and It represents the STFT matrix before and after embedding the watermark, α represents the embedding strength factor, the larger the α is, the better the robustness is, and the smaller the α is, the better the imperceptibility is.

[0109] In this embodiment, the audio frames processed by watermark embedding contain both the original audio information and the watermark signal with dynamically adjusted strength. These frames are then reassembled to generate a complete audio file embedded with the watermark. The whole process ensures that the embedding of the watermark will not cause obvious auditory distortion, and can also maintain the extractability and robustness of the watermark when facing various attacks such as desynchronization attacks, compression, and noise addition.

[0110] The above embodiment is described below by way of experimental verification:

[0111] In order to train the robustness of the network, this method sets a distortion layer to simulate a variety of possible attacks (such as MP3 compression, Gaussian noise, cropping, etc.), and controls the probability and parameters of the attack to train the model's performance in the face of these attacks. The main purpose of setting the distortion layer is to simulate various adverse conditions and attacks that audio may encounter during actual transmission and use, so as to test and enhance the resistance of the audio robust watermarking algorithm. By applying different types and intensities of distortion to the audio data during the training phase, the algorithm can learn how to effectively embed and extract watermark information in the face of these challenges, thereby ensuring the robustness of the watermark.

[0112] The different attack types and parameters in the distortion layer are shown in Table 1:

[0113]

[0114] Table 1

[0115] TSM attack (Time Stretch Modification) is a common desynchronization attack that destroys the synchronization of the watermark by time stretching the audio signal and changing the audio playback speed. However, when training deep learning models, if TSM attacks with random parameters are used, model training will be unstable and affect model performance. In the training phase, this method does not use TSM attacks with random parameters, but selects fixed parameters for attack. On the one hand, it can ensure that the type and intensity of attacks faced by the model are the same during each training process, making the training process more stable. On the other hand, it can reduce the randomness in the training process, so that the model faces a deterministic distortion type in each iteration, thereby helping the model to learn more stably how to combat these specific time domain deformations, avoiding confusion or inconsistency in the learning process caused by frequent parameter changes, and improving robustness.

[0116] In order to verify the effectiveness of this method, an imperceptibility experiment was designed. The method without adding the strength prediction network and the method with adding the embedding strength prediction network were compared on different data sets, and the imperceptibility of watermark embedding was evaluated based on indicators such as signal-to-noise ratio, objective difference level, and perceptual speech quality assessment.

[0117] This method uses LibriSpeech and other datasets to train the model and tests it on different types of audio datasets to ensure that the model can effectively embed and extract watermark information on different audio data and has good robustness and imperceptibility. 2000 speech segments were randomly selected from the LibriSpeech dataset (speech, 16 bits, 16kHz) for training the model. The length of these segments is L A =161,290, which means it contains ten audio frames in total. Another 500 audios are selected as validation sets to verify the performance of the model and ensure that the knowledge learned by the model in the training set can be generalized to new data. In the testing phase, different speech and music datasets are tested to test the performance of the model in actual scenarios and evaluate the robustness and imperceptibility of the model. The speech and music datasets include LibriSpeech, FMA, LJSpeech, VCTK, etc., and the length is set to L A =161,290.

[0118] This method evaluates the imperceptibility of the embedded watermark through indicators such as Signal-to-Noise Ratio (SNR), Objective Difference Grade (ODG) (only for music clips), and Perceptual Evaluation of Speech Quality (PESQ) (only for speech clips). Signal-to-Noise Ratio is used to measure the ratio of the signal to noise of the audio signal before and after the watermark is embedded. The higher the value, the less obvious the watermark is and the less impact it has on the audio quality. Objective Difference Grade is an objective indicator of the difference in sound quality before and after the watermark is embedded in the music clip. The lower the value, the smaller the difference in sound quality. Perceptual Evaluation of Speech Quality can evaluate the perceptual indicators of the speech quality before and after the watermark is embedded in the speech clip. The higher the value, the better the speech quality.

[0119] In addition, in order to better evaluate the performance of the synchronization code, this method uses the frame synchronization error rate (Frame Synchronization Error Rate, FSER) to measure the reliability of the synchronization code:

[0120]

[0121] in represents the actual number of audio frames with watermark payload embedded, M represents the total number of audio samples in the dataset, Represents the number of frames successfully synchronized in the audio through fixed-length synchronization decoding. The upper part of the formula represents the number of audio frames that are missed or misdetected. The FSER indicator effectively evaluates the average success rate of the synchronization code in a given audio data set. The lower the FSER, the better the performance of the synchronization code, and the easier it is to accurately extract the watermark information.

[0122] This method compares the method without adding the intensity prediction network (DRAW) and the method with adding the embedded intensity prediction network (Ada-DRAW). First, the perceptual quality results of the two algorithms are tested on different datasets, where the embedded intensity factor refers to the average intensity factor of all audio frames in the dataset.

[0123] The comparison of imperceptibility on different data sets is shown in Table 2:

[0124]

[0125] Table 2

[0126] As can be seen from Table 2, the perceptual quality of Ada-DRAW on different data sets is better than that of DRAW. Compared with DRAW, which uses the same embedding strength factor of 1 for each different data set, Ada-DRAW can select different embedding strength factors for different audio and audio frames, and improve the imperceptibility of the watermark by selecting the appropriate embedding strength factor.

[0127] like Figure 4 As shown, this method selects two audio segments as visualization, and selects sample points with a length of 161,290, i.e., ten audio frames containing watermarks, where Ada-DRAW outputs different embedding strength factors for different audio frames. Figure 4 The numbers above each frame in the figure correspond to the embedding strength factors obtained by Ada-DRAW through the embedding strength prediction network. It can be seen that for regions with smaller amplitudes (such as Figure 4 In the first two to three frames of the audio, Ada-DRAW tends to predict a smaller strength factor, while in the area with larger amplitude, Ada-DRAW tends to predict a larger strength factor. This is because for the area with smaller amplitude, the imperceptibility can be improved by a smaller embedding strength while maintaining robustness, while for the area with larger amplitude, the robustness can be improved by a larger embedding strength while maintaining a certain imperceptibility. The smaller the embedding strength factor, the better the imperceptibility of the watermark. Conversely, the larger the embedding strength factor, the better the robustness of the watermark. It can be seen that the algorithm can select the appropriate embedding strength factor according to the characteristics of the audio itself, so as to make a more targeted trade-off between imperceptibility and robustness.

[0128] The synchronization code is a key part of the audio robust watermarking technology used to ensure the correctness of watermark information extraction. It is usually embedded in the audio signal and used to indicate the starting position of the watermark information. The synchronization code can locate the watermark information and help the decoder accurately find the starting position of the watermark information so that the watermark content can be correctly extracted. The synchronization code can also resist desynchronization attacks. Since some attack methods (such as cropping, time stretching, etc.) will destroy the synchronization of the watermark information, the decoder cannot correctly extract the watermark, and the synchronization code can help the decoder overcome these attacks and restore the synchronization of the watermark information.

[0129] This method tests the FSER of synchronization code extraction by using only a fixed-length synchronization decoder. FSER is used to measure the ratio of successfully synchronized watermark frames to the total number of frames in an audio dataset and is an effective indicator for evaluating the reliability of synchronization codes. A lower FSER indicates higher synchronization accuracy and the ability to resist attacks.

[0130] The synchronization code performance (expressed in FSER (%)) of Ada-DRAW and DRAW on LibriSpeech is shown in Table 3. The symbol “*” indicates that the attack (or corresponding parameters) is not included during training.

[0131]

[0132]

[0133] Table 3

[0134] Table 3 shows the results when the test data is subjected to various types of attacks. It can be seen that the proposed Ada-DRAW and DRAW have lower synchronization error rates under different attacks, and the synchronization error rate of Ada-DRAW is slightly lower than that of DRAW. The test using only a fixed-length synchronization decoder not only verifies the robustness of Ada-DRAW under various attack conditions, but also highlights its optimization in synchronization code performance compared to DRAW, that is, while maintaining a low synchronization error rate, it further improves its resistance to attacks.

[0135] In addition, this method also tests the performance of Ada-DRAW and DRAW under various common signal processing operations (such as Gaussian noise, MP3 compression, etc.) and various desynchronization attacks (including jitter, random cropping, time stretching modification and pitch scaling). The decoding performance of Ada-DRAW and DRAW on LibriSpeech (expressed in ACC (%)) is shown in Table 4. The symbol "*" indicates that the attack (or corresponding parameter) was not included during training.

[0136]

[0137]

[0138] Table 4

[0139] The experimental results are shown in Table 4. It can be seen that compared with DRAW, Ada-DRAW has improved performance in most attack scenarios. Ada-DRAW improves the robustness of the watermark algorithm while ensuring imperceptibility by adaptively adjusting the embedding strength.

[0140] In order to better illustrate the characteristics of Ada-DRAW, this method analyzes the statistical results of the embedding strength factor of all audio frames on different data sets. Figure 5As shown in the figure, this method gives the results in the form of histograms, which correspond to the statistical results of four datasets: LibriSpeech (upper left), FMA (upper right), LJSpeech (lower left), and VCTK (lower right). On different datasets, the algorithm can fine-tune the intensity factor according to the characteristics of the audio itself, thereby ensuring that the imperceptibility and robustness for specific audio are optimized.

[0141] This embodiment achieves the following technical effects through the above technical solution:

[0142] This embodiment obtains audio data, performs frame division and watermark embedding processing on the audio data, and can generate a corresponding three-dimensional coefficient matrix; then, the three-dimensional coefficient matrix is ​​input into an embedding strength prediction network, and a corresponding embedding strength factor is output; finally, the embedding strength of the watermark is adjusted according to the embedding strength factor, and the audio frame with the watermark information embedded is output, so that the embedding strength is dynamically adjusted according to the characteristics of each audio, which can not only better protect the copyright, but also improve the anti-attack ability of the watermark without affecting the sound quality; the present invention proposes a new method for generating an adaptive audio robust watermark based on an embedding strength prediction network, which can be fine-tuned according to the characteristics of different audios so that it can adapt to various types of audio content; and selects a suitable embedding strength according to the characteristics of different audio frames, so that the watermark information can be better hidden and the imperceptibility can be improved without affecting the sound quality; by adaptively adjusting the embedding strength, while ensuring imperceptibility, the robustness can be improved as much as possible, so that it can resist various signal processing operations and desynchronization attacks; at the same time, it can make a trade-off between imperceptibility and robustness, and select a suitable embedding strength according to the specific application scenario, so as to achieve the best copyright protection effect.

[0143] Exemplary Devices

[0144] Based on the above embodiments, the present invention also provides a terminal, comprising: a processor, a memory, an interface, a display screen and a communication module connected via a system bus; wherein the processor is used to provide computing and control capabilities; the memory includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect external devices, such as mobile terminals and computers; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0145] When the computer program is executed by the processor, it is used to implement the operation of a method for generating an audio robust watermark based on an embedded strength prediction network.

[0146] It can be understood by those skilled in the art that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores an audio robust watermark generation program based on an embedded strength prediction network, and the audio robust watermark generation program based on an embedded strength prediction network is used to implement the operation of the audio robust watermark generation method based on an embedded strength prediction network as described above when executed by the processor.

[0148] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores an audio robust watermark generation program based on an embedded strength prediction network, and when the audio robust watermark generation program based on an embedded strength prediction network is executed by the processor, it is used to implement the operation of the audio robust watermark generation method based on the embedded strength prediction network as described above.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory.

[0150] In summary, the present invention provides an audio robust watermark generation method, terminal and storage medium based on an embedding strength prediction network. The method includes: obtaining audio data, framing and watermark embedding the audio data, and generating a corresponding three-dimensional coefficient matrix; inputting the three-dimensional coefficient matrix into an embedding strength prediction network, and outputting a corresponding embedding strength factor; adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame embedded with watermark information; the present invention proposes a new method for adaptive audio robust watermark generation based on an embedding strength prediction network, which improves the imperceptibility and robustness of the watermark algorithm.

[0151] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for generating robust audio watermark based on embedded strength prediction network, characterized in that: include: Acquire audio data, perform frame segmentation and watermark embedding processing on the audio data, and generate a corresponding three-dimensional coefficient matrix; Inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor; The embedding strength of the watermark is adjusted according to the embedding strength factor, and an audio frame embedded with the watermark information is output.

2. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 1, characterized in that: The step of acquiring audio data, performing frame segmentation and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix includes: Acquire the audio data, perform frame processing on the audio data, and perform short-time Fourier transform processing on each frame of audio after the frame division to obtain a corresponding two-dimensional coefficient matrix; The real part and the imaginary part of the two-dimensional coefficient matrix are concatenated and watermark embedded to obtain the three-dimensional coefficient matrix.

3. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 2, characterized in that: The short-time Fourier transform is performed on each frame of audio after framing to obtain a corresponding two-dimensional coefficient matrix, including: Weighting is performed by multiplying the sample points of each frame of audio after framing by the corresponding value of the Hanning window function; The corresponding two-dimensional coefficient matrix is ​​obtained by short-time Fourier transform for each frame of audio after windowing.

4. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 1, characterized in that: The step of inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor comprises: Input the three-dimensional coefficient matrix into the embedding strength prediction network, and obtain the embedding strength factor corresponding to each frame of audio through feature concatenation and mapping processing; The embedding strength prediction network outputs the embedding strength factor corresponding to each frame of audio.

5. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 4, characterized in that: The step of inputting the three-dimensional coefficient matrix into the embedding strength prediction network and obtaining the embedding strength factor corresponding to each frame of audio through feature concatenation and mapping processing includes: Inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix; Concatenate the feature representation matrix and the two-dimensional coefficient matrix on the channel and map them back to the initial number of channels; The embedding strength factor is determined according to the mapped feature representation matrix.

6. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 5, characterized in that: The step of inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix comprises: The three-dimensional coefficient matrix is ​​sequentially input into a convolution module and four channel attention compression and excitation modules for feature enhancement to obtain a corresponding feature representation matrix.

7. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 5, characterized in that: The step of concatenating the feature representation matrix with the two-dimensional coefficient matrix on the channel and mapping back to the initial number of channels includes: The feature representation matrix and the two-dimensional coefficient matrix are concatenated on the channel, and mapped back to the initial number of channels through a convolution module and a pooling module; The output range of the feature representation matrix is ​​mapped to the 0-1 interval through the sigmoid activation function.

8. The method for generating an audio robust watermark based on an embedded strength prediction network according to claim 1, characterized in that: The step of adjusting the embedding strength of the watermark according to the embedding strength factor and outputting the audio frame embedded with the watermark information comprises: adjusting the embedding strength of the watermark according to the embedding strength factor, and embedding watermark information into the audio frame according to the embedding strength of the watermark; The audio frame with the embedded watermark information is output.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores an audio robust watermark generation program based on an embedded strength prediction network, and when the audio robust watermark generation program based on an embedded strength prediction network is executed by the processor, it is used to implement the operation of the audio robust watermark generation method based on an embedded strength prediction network as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an audio robust watermark generation program based on an embedded strength prediction network, and when the audio robust watermark generation program based on an embedded strength prediction network is executed by a processor, it is used to implement the operation of the audio robust watermark generation method based on an embedded strength prediction network as described in any one of claims 1-8.

Images