Audio robust watermark generation method and terminal based on embedding strength prediction network

Abstract

The application discloses an audio robust watermark generation method and a terminal based on an embedding strength prediction network, and the method comprises the following steps: acquiring audio data, performing frame division and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix; inputting the three-dimensional coefficient matrix into an embedding strength prediction network, and outputting a corresponding embedding strength factor; adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame embedded with watermark information; the application proposes a new self-adaptive audio robust watermark generation method based on an embedding strength prediction network, and improves the imperceptibility and robustness of the watermark algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio robust watermark technology, and in particular to an audio robust watermark generation method based on an embedding strength prediction network and a terminal. BACKGROUND

[0002] The rapid development of the Internet has led to the widespread dissemination of digital media, among which audio occupies an important position in daily life and social communication due to its convenient communication method. The popularity of audio editing software and portable recording equipment has simplified the production and sharing of audio, but has also brought copyright and privacy security problems. Ensuring the authenticity of audio and copyright protection has become an urgent problem to be solved. In recent years, the emerging technology of deep learning has been widely applied to traditional digital robust watermark technology for copyright protection, which has attracted widespread attention. Audio robust watermarking is a technology that embeds and extracts corresponding information in audio signals, aiming to protect the integrity and copyright of audio content. Audio robust watermarking algorithm based on deep learning has become an important research hotspot, and it is of great significance to conduct in-depth research on it.

[0003] However, existing audio watermarking methods based on deep learning generally lack consideration of embedding strength differences when processing different audios. These algorithms are usually trained on large datasets and use the same embedding strength for all audios in practical applications. However, for individual audios, a fixed embedding strength is difficult to optimize both transparency and robustness. Each audio has different characteristics, and a fixed strength may cause the watermark to be too obvious in some audios, affecting the auditory experience, or not obvious enough in some audios, reducing robustness.

[0004] Therefore, the prior art still needs to be improved. SUMMARY

[0005] The technical problem to be solved by the present application is that, in view of the defects of the prior art, the present application provides an audio robust watermark generation method based on an embedding strength prediction network, a terminal and a storage medium, to solve the technical problem of lack of dynamic adjustment of embedding strength in traditional deep learning-based audio watermarking algorithms.

[0006] The technical solution adopted by the present application to solve the technical problem is as follows:

[0007] In a first aspect, the present application provides an audio robust watermark generation method based on an embedding strength prediction network, comprising:

[0008] obtaining audio data, performing frame and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix;

[0009] input the three-dimensional coefficient matrix into the embedding strength prediction network, and output a corresponding embedding strength factor;

[0010] adjust the embedding strength of the watermark according to the embedding strength factor, and output an audio frame in which watermark information is embedded.

[0011] In an implementation manner, the obtaining audio data, performing frame processing and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix, comprises:

[0012] obtaining the audio data, performing frame processing on the audio data, and performing short-time Fourier transform processing on each frame of audio after frame processing to obtain a corresponding two-dimensional coefficient matrix;

[0013] splicing the real part and the imaginary part of the two-dimensional coefficient matrix, and performing watermark embedding processing to obtain the three-dimensional coefficient matrix.

[0014] In an implementation manner, the performing short-time Fourier transform processing on each frame of audio after frame processing to obtain a corresponding two-dimensional coefficient matrix, comprises:

[0015] multiplying sample points of each frame of audio after frame processing by corresponding values of the Hanning window function for weighting;

[0016] obtaining a corresponding two-dimensional coefficient matrix of each frame of audio after windowing through short-time Fourier transform.

[0017] In an implementation manner, the inputting the three-dimensional coefficient matrix into the embedding strength prediction network, and outputting a corresponding embedding strength factor, comprises:

[0018] inputting the three-dimensional coefficient matrix into the embedding strength prediction network, obtaining a corresponding embedding strength factor of each frame of audio through feature splicing and mapping processing;

[0019] outputting a corresponding embedding strength factor of each frame of audio through the embedding strength prediction network.

[0020] In an implementation manner, the inputting the three-dimensional coefficient matrix into the embedding strength prediction network, obtaining a corresponding embedding strength factor of each frame of audio through feature splicing and mapping processing, comprises:

[0021] inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix;

[0022] splicing the feature representation matrix and the two-dimensional coefficient matrix in the channel, and mapping back to the initial number of channels;

[0023] determining the embedding strength factor according to the mapped feature representation matrix.

[0024] In an implementation manner, the inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain corresponding feature representation matrices comprises:

[0025] The three-dimensional coefficient matrix is sequentially input into a convolution module and a compression and excitation module of four channel attentions for feature enhancement to obtain corresponding feature representation matrices.

[0026] In an implementation manner, the splicing the feature representation matrix and the two-dimensional coefficient matrix in the channel and mapping back to the initial channel number comprises:

[0027] The feature representation matrix and the two-dimensional coefficient matrix are spliced in the channel, and are mapped back to the initial channel number through a convolution module and a pooling module.

[0028] The output range of the feature representation matrix is mapped to the interval of 0-1 through a sigmoid activation function.

[0029] In an implementation manner, the adjusting the embedding strength of the watermark according to the embedding strength factor and outputting an audio frame embedded with watermark information comprises:

[0030] The embedding strength of the watermark is adjusted according to the embedding strength factor, and the audio frame is embedded with watermark information according to the embedding strength of the watermark.

[0031] The audio frame embedded with watermark information is output.

[0032] In a second aspect, the present application further provides a terminal comprising a processor and a memory, wherein the memory stores an audio robust watermark generation program based on an embedding strength prediction network, and the audio robust watermark generation program based on the embedding strength prediction network is used to implement the operations of the audio robust watermark generation method based on the embedding strength prediction network when executed by the processor.

[0033] In a third aspect, the present application further provides a computer readable storage medium storing an audio robust watermark generation program based on an embedding strength prediction network, and the audio robust watermark generation program based on the embedding strength prediction network is used to implement the operations of the audio robust watermark generation method based on the embedding strength prediction network when executed by a processor.

[0034] The present application has the following effects by adopting the above technical solutions:

[0035] The application can generate corresponding three-dimensional coefficient matrixes by acquiring audio data and performing frame and watermark embedding processing on the audio data; then, input the three-dimensional coefficient matrixes into an embedding strength prediction network and output corresponding embedding strength factors; finally, adjust the embedding strength of the watermark according to the embedding strength factors and output audio frames with embedded watermark information, so that the embedding strength is dynamically adjusted according to the characteristics of each audio, which not only better protects the copyright, but also improves the anti-attack ability of the watermark without affecting the sound quality; the application proposes a new adaptive audio robust watermark generation method based on an embedding strength prediction network, which improves the imperceptibility and robustness of the watermark algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the drawings shown.

[0037] Figure 1 is a flow chart of an audio robust watermark generation method based on an embedding strength prediction network in an implementation manner of the present application.

[0038] Figure 2 is a structural schematic diagram of an adaptive audio robust watermark generation network based on an embedding strength prediction network in an implementation manner of the present application.

[0039] Figure 3 is a structural schematic diagram of an embedding strength prediction network in an implementation manner of the present application.

[0040] Figure 4 is a schematic diagram of embedding strength factors of different audios in an implementation manner of the present application.

[0041] Figure 5 is a schematic diagram of embedding strength prediction for different data sets in an implementation manner of the present application.

[0042] Figure 6 is a functional principle diagram of a terminal in an implementation manner of the present application.

[0043] The implementation of the present application, functional characteristics and advantages will be further described with reference to the drawings. DETAILED DESCRIPTION

[0044] The application provides an audio robust watermark generation method based on an embedding strength prediction network, a terminal and a storage medium. In order to make the purpose, technical scheme and advantages of the application more clear and explicit, the application will be further described in detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and not to limit the application.

[0045] Exemplary method

[0046] Since the existing deep learning-based audio watermarking methods generally lack consideration of embedding strength differences when processing different audios. These algorithms are usually trained on large datasets and use the same embedding strength for all audios in practical applications. However, for individual audios, a fixed embedding strength is difficult to optimize transparency and robustness at the same time. Each audio has different characteristics, and a fixed strength may cause the watermark to be too obvious in some audios, affecting the auditory experience, or not obvious enough in some audios, reducing robustness. Therefore, in order to achieve the best transparency and robustness, it is necessary to dynamically adjust the embedding strength according to the characteristics of each audio. This dynamic adjustment not only better protects copyright, but also improves the anti-attack ability of the watermark without affecting the sound quality.

[0047] Although deep learning-based audio watermarking technology shows great potential in copyright protection, the embedding strength optimization problem in practical applications still needs further research and improvement. By implementing dynamic adjustment of embedding strength at the level of individual audios, better balance between transparency and robustness can be achieved, promoting the development of this field.

[0048] To solve the above technical problems, the embodiment of the application provides an audio robust watermark generation method based on an embedding strength prediction network, which aims to propose a new adaptive audio robust watermark generation method based on an embedding strength prediction network, and improve the imperceptibility and robustness of the watermark algorithm.

[0049] As shown in Figure 1 The embodiment of the application provides an audio robust watermark generation method based on an embedding strength prediction network, which includes the following steps:

[0050] Step S100, acquiring audio data, performing frame and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix.

[0051] In this embodiment, the audio robust watermark generation method based on the embedding strength prediction network is applied to a terminal, and the terminal includes but is not limited to a computer, a mobile terminal and other devices; the terminal is provided with a training migration platform of the audio robust watermark generation model based on the embedding strength prediction network.

[0052] In the embodiment, a method of adaptive audio robust watermark generation based on embedding strength prediction network is proposed. By introducing an embedding strength prediction network, suitable embedding strength is selected for different audio frames, further improving the imperceptibility and robustness of the watermark algorithm, so as to output suitable embedding strength for different audio frames to improve the performance of the algorithm. This dynamic adjustment not only better protects the copyright, but also improves the anti-attack ability of the watermark without affecting the sound quality.

[0053] As shown in Figure 2 The application adds an embedding strength prediction network based on the codec. The network acts between the encoder and the distortion layer, takes each audio frame as input, and outputs an embedding strength factor. By selecting suitable parameters for each audio frame, the imperceptibility and robustness are improved. The specific structure of the network is shown in Figure 2 The overall framework of the network includes an encoder, a distortion layer, a decoder, and an embedding strength prediction network.

[0054] In the research and application of audio watermarking technology, embedding strength plays a crucial role. Essentially, the embedding strength prediction network is more like an auxiliary network structure. Its characteristic is independent of the original backbone network (i.e., encoder-distortion layer-decoder) and can be used as a plug-in. The main responsibility of the embedding strength prediction network is to provide the encoder with an optimal embedding strength factor selection based on the content of the audio itself and the watermark information to be embedded. The so-called "optimal" strength factor refers to the factor that, under its adjustment, the decoder can accurately and accurately parse the secret information hidden in the watermark, while also trying to reduce the strength required in the watermark embedding process, thereby maximizing the potential of the audio without increasing the bit error rate, achieving high concealment of the watermark. In other words, the original intention of the design of the embedding strength prediction network is to fully exploit and utilize the characteristics of the audio itself without sacrificing audio quality and increasing the bit error rate, in order to achieve the dual purpose of effectively hiding watermark information and maintaining audio quality.

[0055] In the embodiment, first, audio data is read from the original audio file. In order to facilitate processing and analysis, the audio data is divided into a series of non-overlapping short-time frames. Short-time Fourier transform is applied to each audio frame to obtain a three-dimensional real-valued STFT coefficient matrix. This three-dimensional coefficient matrix contains the frequency domain information of the audio and is the basis for watermark embedding and extraction.

[0056] Specifically, in one implementation manner of the embodiment, step S100 includes the following steps:

[0057] Step S110, obtaining the audio data, performing frame processing on the audio data, and performing short-time Fourier transform processing on each frame of audio after frame processing to obtain a corresponding two-dimensional coefficient matrix;

[0058] Step S120, splicing the real part and the imaginary part of the two-dimensional coefficient matrix, and performing watermark embedding processing to obtain the three-dimensional coefficient matrix.

[0059] In the embodiment, first, audio data is read from an original audio file. The audio segments can be randomly selected from LibriSpeech, FMA, LJSpeech, VCTK, etc. The length of these segments is set to 161,290, that is, a total of ten audio frames. For the input audio file, the application adopts a frame-by-frame embedding manner. First, the audio is divided into mutually non-overlapping frames with a fixed length, and subsequent watermark embedding and extraction are performed. For the length L A Carrier audio A = {A(a) | 1≤a≤L A}, A(a) represents the a-th sample point, and A is divided into n frames A1, A2, …, An with a frame length of L F (For example, L F = 16129) without overlapping. n For the frame with less than L F sample points at the end of the audio, the watermark is not embedded.

[0060] In the embodiment, for each frame of audio A i (i = 1, …, n) after frame processing, a short-time Fourier transform is performed on it with a window length of N and a window shift of L h to obtain a two-dimensional coefficient matrix The matrix size after the STFT transform is represented as S

[0061] In the embodiment, the real part and the imaginary part of the two-dimensional coefficient matrix S i are spliced on the channel, and the spliced matrix is subjected to watermark embedding to obtain a three-dimensional coefficient matrix which is used as the input of the subsequent encoder and embedding strength prediction network. The purpose of this is to facilitate the subsequent deep learning model processing, that is, to convert the complex number data into real number data format that is easy for the neural network to process.

[0062] Specifically, in one implementation manner of the embodiment, step S110 includes the following steps:

[0063] Step S111, multiplying the sample points of each frame of audio after frame processing by the corresponding value of the Hanning window function for weighting;

[0064] Step S112: Obtain the corresponding two-dimensional coefficient matrix by performing a short-time Fourier transform on each frame of audio after windowing.

[0065] In this embodiment, the windowing process is an important step in digital signal processing, especially when performing short-time Fourier transforms. The audio signal is divided into a series of non-overlapping or partially overlapping short-time windows, and then a Fast Fourier Transform (FFT) is applied to each window to obtain the spectral information within that time period. Windowing can reduce boundary effects and smooth frame edges.

[0066] In this embodiment, for each frame of audio A after framing... i (i = 1, ..., n), and construct a window of length N and a window shift of L. h The short-time Fourier transform yields the corresponding two-dimensional complex matrix S. i :

[0067]

[0068] in, This represents the index value of the window. Let g(c) represent the frequency index and g(c) represent the Hanning window function.

[0069]

[0070] In this embodiment, the sample points of each frame of audio after framing are multiplied by the corresponding value of the Hanning window function and weighted. The windowing step occurs after the audio signal is segmented into frames and before STFT is performed, which ensures that the processing of each audio frame can effectively reduce edge effects and obtain more accurate spectrum analysis results, which is crucial for subsequent robust audio watermark embedding.

[0071] In this embodiment, the audio signal is first segmented into non-overlapping frames, which facilitates subsequent frame-by-frame processing; and watermarking is not performed on frames with insufficient audio samples at the end to avoid edge effects; each frame of audio obtains its spectral representation through short-time Fourier transform (STFT), and the spectrum is smoothed using the Hanning window function to ensure the accuracy of the transform.

[0072] like Figure 1 As shown, in one implementation of this invention, the audio robust watermark generation method based on an embedding strength prediction network further includes the following steps:

[0073] Step S200: Input the three-dimensional coefficient matrix into the embedding strength prediction network and output the corresponding embedding strength factor.

[0074] In this embodiment, the structure of the embedded strength prediction network is as follows: Figure 3The three-dimensional short-time Fourier transform coefficient matrix after watermark embedding is shown As input, an embedding strength factor a is output. The embedding strength prediction network provides a customized embedding strength factor for each frame by deep analysis of the audio frame, to optimize the embedding process of the watermark and ensure the best performance balance under different audio content and conditions.

[0075] Specifically, in an implementation manner of the embodiment, step S200 includes the following steps:

[0076] Step S210, inputting the three-dimensional coefficient matrix into the embedding strength prediction network to obtain the embedding strength factor corresponding to each audio frame through feature splicing and mapping processing;

[0077] Step S220, outputting the embedding strength factor corresponding to each audio frame through the embedding strength prediction network.

[0078] In the embodiment, the embedding strength prediction network includes a plurality of feature extraction modules, which extract features from the input three-dimensional coefficient matrix to obtain higher-level feature representations. Through a plurality of such modules, the network can capture the features of the audio frame while keeping the size of the feature map unchanged. The extracted feature representations are spliced with the original three-dimensional coefficient matrix in the channel to form a feature matrix containing more information. The spliced feature matrix is mapped back to the initial number of channels through network layers such as convolution layers, ensuring efficient integration and transmission of information. After the mapping processing, the embedding strength prediction network outputs a value between 0 and 1, representing the embedding strength factor of the audio frame. This embedding strength factor will be used to adjust the strength of the watermark, so as to balance between imperceptibility and robustness.

[0079] Specifically, in an implementation manner of the embodiment, step S210 includes the following steps:

[0080] Step S211, inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix;

[0081] Step S212, splicing the feature representation matrix with the two-dimensional coefficient matrix in the channel and mapping back to the initial number of channels;

[0082] Step S213, determining the embedding strength factor according to the mapped feature representation matrix.

[0083] In the present embodiment, in the feature extraction part, a compression and excitation module (Squeeze-and-Excitation block, SE block) based on channel attention is selected as the basic module. The SE block can effectively capture the importance of features and recalibrate the weights, thereby improving the network's attention to key information. For the input three-dimensional coefficient matrix Through four feature extraction modules, the shape and size are kept unchanged while the features are extracted, and the corresponding feature representation is obtained

[0084]

[0085] In the present embodiment, the feature representation is combined with the two-dimensional coefficient matrix S i After splicing in the channel, it is mapped back to the initial channel number through the convolution layer, and the output range is mapped to the 0-1 interval through the sigmoid activation function:

[0086]

[0087] In the present embodiment, in order to ensure the stability of network training, the output embedding strength factor a can be controlled in the range of 0.7 to 1.3, ensuring that the watermark has sufficient robustness without compromising the quality of the audio.

[0088]

[0089] In the present embodiment, the output embedding strength factor is then used to adjust the embedding process of the watermark. Each audio frame adjusts the embedding degree according to its content and the predicted strength factor, realizing the individualization of the watermark depth.

[0090] Specifically, in one implementation manner of the present embodiment, step S211 includes the following steps:

[0091] Step S211a, inputting the three-dimensional coefficient matrix into a convolution module and four channel attention compression and excitation modules in sequence for feature enhancement, to obtain a corresponding feature representation matrix.

[0092] In this embodiment, the constructed three-dimensional coefficient matrix is sent into a network structure composed of a convolution module and four compression and excitation modules based on channel attention mechanism. First, the network uses one or more convolution layers to process the three-dimensional coefficient matrix, performing preliminary feature extraction. The convolution layer can capture local spatiotemporal features, providing a basis for subsequent feature enhancement. Then, the features are further enhanced through four consecutive SE modules. The core function of the SE module is to "squeeze" (Squeeze) the feature map through global average pooling, reducing the dimension to the channel dimension, and then "excitation" (Excitation) through two fully connected layers to learn the importance weight of each channel. These weights are used to recalibrate the previous features, emphasizing the channels that are more important to the task, thereby achieving selective enhancement of features. After the above convolution module and channel attention module processing, the features of each audio frame are significantly enhanced, forming a more refined and discriminative feature representation matrix. This matrix not only retains the key information of the audio frame, but also highlights the parts that are conducive to predicting the embedding strength.

[0093] Specifically, in an implementation manner of the embodiment, step S212 includes the following steps:

[0094] Step S212a, concatenating the feature representation matrix and the two-dimensional coefficient matrix in the channel, and mapping back to the initial number of channels through a convolution module and a pooling module;

[0095] Step S212b, mapping the output range of the feature representation matrix to the 0-1 interval through a sigmoid activation function.

[0096] In this embodiment, first, the feature representation matrix obtained through the channel attention compression and excitation module is concatenated with the original two-dimensional complex coefficient matrix in the channel dimension. Thus, the high-level abstract feature information is combined with the original frequency domain representation of the audio, so that the model can better understand the characteristics of each audio frame and make more accurate strength prediction accordingly. The concatenated matrix is then sent to a convolution module, which may include multiple convolution layers for further extracting and integrating key information in the concatenated features. The convolution module helps to capture local features and patterns, and the pooling module is used to reduce the spatial dimension of the feature map while retaining the most important feature information, thereby reducing computational complexity and enhancing robustness to spatial changes. This process finally maps the features back to the initial number of channels, ensuring compatibility with subsequent processing.

[0097] In this embodiment, the features obtained through the above processing are convolved and pooled, and then activated by a sigmoid function. The sigmoid function is an S-shaped function whose output range is strictly limited to (0,1), suitable for probability estimation or scenarios where the output value needs to be constrained to a specific range. In this application, it is used to map the output of the feature representation matrix to the range of 0 to 1, obtaining an embedding strength factor for each audio frame. These factors directly indicate the strength to be used when embedding the watermark in each frame to achieve the best balance between transparency and robustness. A larger output value (close to 1) indicates a higher embedding strength, meaning the watermark is more robust but may have a greater impact on audio quality; conversely, a smaller value (close to 0) indicates a weaker embedding, improving imperceptibility but potentially sacrificing some robustness.

[0098] In this embodiment, an embedding strength prediction network is used to dynamically adjust the embedding strength of the audio watermark, improving its imperceptibility and robustness. A three-dimensional coefficient matrix is ​​input into the embedding strength prediction network, which extracts features through multiple feature extraction modules and a channel attention mechanism. These features are then concatenated with the two-dimensional coefficient matrix and mapped back to the initial number of channels. Finally, an embedding strength factor in the 0-1 range is output through a sigmoid activation function. This dynamic adjustment of the watermark embedding strength ensures that the transparency and robustness of the robust audio watermark are optimal.

[0099] like Figure 1 As shown, in one implementation of this invention, the audio robust watermark generation method based on an embedding strength prediction network further includes the following steps:

[0100] Step S300: Adjust the embedding intensity of the watermark according to the embedding intensity factor, and output the audio frame containing the embedded watermark information.

[0101] In this embodiment, by predicting the embedding strength factor individually for each audio frame, the embedding strength of the watermark can be dynamically adjusted according to the characteristics of the audio content. For noisy or complex parts of the audio, the algorithm can predict a smaller embedding strength to ensure the watermark's imperceptibility, making the watermark almost unaffected by the original audio's sound quality. Conversely, for stable or strong signal parts of the audio, a larger embedding strength factor can be used to ensure that the watermark can still be accurately extracted even after an attack, improving the watermark's robustness. The encoder uses the adjusted embedding strength factor to embed the watermark information into the audio frame. Subsequently, whether in the original audio or audio subjected to various attacks, the decoder can attempt to extract the watermark information to verify the audio's integrity and copyright ownership.

[0102] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0103] Step S310, adjusting the embedding strength of the watermark according to the embedding strength factor, and embedding the watermark information into the audio frame according to the embedding strength of the watermark;

[0104] Step S320, outputting the audio frame in which the watermark information is embedded.

[0105] In this embodiment, the strength of the watermark signal is adjusted by the embedding strength factor (a) according to the characteristics of each audio frame. This means that the strength of the watermark embedding is not fixed, but is adjusted according to the dynamic changes of the audio content, so as to maximize the hiding property and the ability to resist attacks of the watermark without significantly affecting the listening experience.

[0106] In this embodiment, the input embedding strength factor is used to embed the watermark information into the audio frame by the following formula, and the strength of the watermark is controlled by the embedding strength factor, so as to balance between imperceptibility and robustness:

[0107]

[0108] wherein S i and represents the STFT matrix before and after embedding the watermark, a represents the embedding strength factor, the larger a is, the better the robustness is, and the smaller a is, the better the imperceptibility is.

[0109] In this embodiment, the audio frame after the watermark embedding process contains both the original audio information and the watermark signal embedded with dynamically adjusted strength. These frames are then recombined to generate a complete audio file with embedded watermark. The entire process ensures that the embedding of the watermark does not cause significant auditory distortion, and also maintains the extractability and robustness of the watermark when facing various attacks such as desynchronization attack, compression, noise addition, etc.

[0110] The above embodiment is described below by way of experimental verification:

[0111] In order to train the robustness of the network, the method sets a distortion layer to simulate various possible attacks (such as MP3 compression, Gaussian noise, cropping, etc.), and controls the probability and parameters of the attacks to train the performance of the model when facing these attacks. The main purpose of setting the distortion layer is to simulate various adverse conditions and attacks that the audio may encounter in the actual propagation and use process, so as to test and enhance the resistance ability of the audio robust watermarking algorithm. By applying different types and strengths of distortion to the audio data in the training stage, the algorithm can learn how to effectively embed and extract watermark information when facing these challenges, thereby ensuring the robustness of the watermark.

[0112] The different attack types and parameters in the distortion layer are shown in Table 1:

[0113]

[0114] Table 1

[0115] TSM attack (Time Stretch Modification) is a common desynchronization attack that changes the playback speed of audio signals by time stretching, thereby destroying the synchronization of the watermark. However, when training a deep learning model, if a TSM attack with random parameters is used, the model training will be unstable, affecting the model performance. In the training phase, this method does not use a TSM attack with random parameters, but instead selects fixed parameters for the attack. On the one hand, it can ensure that the type and strength of the attack faced by the model during each training process are the same, making the training process more stable. On the other hand, it can reduce the randomness in the training process, so that the model faces deterministic distortion types at each iteration, helping the model to more stably learn how to resist these specific time-domain deformations, avoiding confusion or inconsistency in the learning process caused by frequent parameter changes, and improving robustness.

[0116] To verify the effectiveness of this method, an imperceptibility experiment is designed. The method without adding an intensity prediction network and the method with adding an embedded intensity prediction network are compared on different data sets. The use of signal-to-noise ratio, objective difference grade, and perceptual speech quality evaluation indicators are used to evaluate the imperceptibility of watermark embedding.

[0117] This method uses LibriSpeech and other data sets to train the model, and tests it on different types of audio data sets to ensure that the model can effectively embed and extract watermark information on different audio data, with good robustness and imperceptibility. 2000 speech segments were randomly selected from the LibriSpeech data set (speech, 16-bit, 16kHz) for training the model, and the length of these segments was L A = 161,290, i.e. a total of ten audio frames, while another 500 audio were selected as a validation set to verify the performance of the model, ensuring that the knowledge learned by the model on the training set can be generalized to new data. In the testing phase, testing was done on different speech and music data sets to test the performance of the model in real-world scenarios and evaluate the robustness and imperceptibility of the model. The speech and music data sets include LibriSpeech, FMA, LJSpeech, VCTK, etc., and the length is set to L A = 161,290.

[0118] The method evaluates the imperceptibility of the embedded watermark through indicators such as signal-to-noise ratio (SNR), objective difference grade (ODG) (only for music segments), and perceptual evaluation of speech quality (PESQ) (only for speech segments). The signal-to-noise ratio is used to measure the ratio of signal to noise before and after watermark embedding, and the higher the value, the less obvious the watermark and the smaller the impact on audio quality. The objective difference grade evaluation is an objective indicator of the difference in audio quality before and after watermark embedding in music segments, and the lower the value, the smaller the difference in audio quality. The perceptual evaluation of speech quality can evaluate the perceptual indicators of speech quality before and after watermark embedding in speech segments, and the higher the value, the better the speech quality.

[0119] In addition, in order to better evaluate the performance of the synchronization code, the method uses the frame synchronization error rate (FSER) to measure the reliability of the synchronization code:

[0120]

[0121] wherein represents the actual number of audio frames with watermark payload embedded, and M represents the total number of audio samples in the data set, represents the number of frames successfully synchronized in the audio through fixed-length synchronization decoding. The upper part of the formula represents the number of audio frames that are missed or incorrectly detected. The FSER indicator effectively evaluates the average success rate of the synchronization code in a given audio data set, and the lower the FSER, the better the performance of the synchronization code and the easier the watermark information is to be accurately extracted.

[0122] The method compares the method without adding an intensity prediction network (DRAW) and the method with an embedded intensity prediction network (Ada-DRAW). First, test the perceptual quality results of the two algorithms on different data sets, where the embedding intensity factor refers to the average value of the intensity factor of all audio frames in the data set.

[0123] The imperceptibility comparison on different data sets is shown in Table 2:

[0124]

[0125] Table 2

[0126] As can be seen from Table 2, the perceptual quality of Ada-DRAW is better than DRAW on different data sets. Compared with DRAW, which uses the same embedding strength factor of 1 for each data set, Ada-DRAW can select different embedding strength factors for different audio and audio frames, and improve the imperceptibility of the watermark by selecting appropriate embedding strength factors.

[0127] As shown in Figure 4 , the method selects two segments of audio as visual display, selects a sample point with a length of 161,290, that is, ten audio frames containing a watermark, and Ada-DRAW outputs different embedding strength factors for different audio frames. Figure 4 The numbers above each frame in Figure 4 correspond to the embedding strength factors obtained by Ada-DRAW through the embedding strength prediction network. It can be seen that for the area with smaller amplitude (such as the first two to three frames in Figure 4 ), Ada-DRAW tends to predict a smaller strength factor, and for the area with larger amplitude, Ada-DRAW tends to predict a larger strength factor. This is because for the area with smaller amplitude, the imperceptibility can be improved on the premise of maintaining robustness by using smaller embedding strength, and for the area with larger amplitude, the robustness can be improved on the premise of maintaining a certain imperceptibility by using larger embedding strength. Smaller embedding strength factor means better imperceptibility of the watermark, and on the contrary, larger embedding strength factor can also improve the robustness of the watermark. It can be seen that the algorithm can select appropriate embedding strength factors according to the characteristics of the audio itself, so as to better balance the imperceptibility and robustness.

[0128] The synchronization code is a key part of audio robust watermarking technology used to ensure the correctness of watermark information extraction. It is usually embedded in the audio signal and used to indicate the starting position of the watermark information. The synchronization code can locate the watermark information and help the decoder accurately find the starting position of the watermark information, so as to correctly extract the watermark content. The synchronization code can also resist desynchronization attacks, because some attack methods (such as cropping, time stretching, etc.) will destroy the synchronization of the watermark information, causing the decoder to fail to correctly extract the watermark, and the synchronization code can help the decoder overcome these attacks and restore the synchronization of the watermark information.

[0129] The method tests the FSER of the synchronization code extraction by using only a fixed-length synchronization decoder. FSER is used to measure the proportion of successfully synchronized watermark frames to the total number of frames in the audio data set, and is an effective indicator for evaluating the reliability of the synchronization code. Lower FSER indicates higher synchronization accuracy and attack resistance.

[0130] The synchronization code performance (in terms of FSER (%)) of Ada-DRAW and DRAW on LibriSpeech is shown in Table 3. The symbol "*" indicates that the attack (or the corresponding parameter) is not included during training.

[0131]

[0132]

[0133] Table 3

[0134] Table 3 shows the results when the test data is subjected to various types of attacks. It can be seen that the proposed Ada-DRAW has lower synchronization error rates than DRAW under different attacks. By testing only using a fixed-length synchronization decoder, not only is the robustness of Ada-DRAW under various attack conditions verified, but also its optimization in synchronization code performance compared to DRAW is highlighted, i.e., while maintaining a low synchronization error rate, the resistance to attacks is further improved.

[0135] In addition, the method also tests the performance of Ada-DRAW and DRAW under various common signal processing operations (such as Gaussian noise, MP3 compression, etc.) and various desynchronization attacks (including jitter, random cropping, time stretching modification, and pitch scaling). The decoding performance of Ada-DRAW and DRAW on LibriSpeech (in terms of ACC (%)) is shown in Table 4. The symbol "*" indicates that the attack (or the corresponding parameter) is not included during training.

[0136]

[0137]

[0138] Table 4

[0139] The experimental results are shown in Table 4, and it can be seen that Ada-DRAW has improved performance in most attack scenarios compared to DRAW. Ada-DRAW adjusts the embedding strength adaptively, while ensuring imperceptibility, and improves the robustness of the watermarking algorithm.

[0140] In order to better illustrate the characteristics of Ada-DRAW, the method analyzes the statistical results of the embedding strength factor of the algorithm for all audio frames on different data sets. As shown in Table 5, the embedding strength factor of Ada-DRAW is much smaller than DRAW, which means that Ada-DRAW can embed more information in the same amount of data, and thus has better performance in terms of robustness. Figure 5As shown, the method gives the results in the form of histograms, which correspond to the statistical results of the four data sets LibriSpeech (top left), FMA (top right), LJSpeech (bottom left) and VCTK (bottom right), respectively. On different data sets, the algorithm can fine-tune the intensity factor according to the characteristics of the audio itself, so as to ensure that the imperceptibility and robustness for specific audio are optimal.

[0141] The embodiment achieves the following technical effects through the above technical solutions:

[0142] The embodiment can generate a corresponding three-dimensional coefficient matrix by acquiring audio data and performing frame division and watermark embedding processing on the audio data; then, input the three-dimensional coefficient matrix into the embedding intensity prediction network and output a corresponding embedding intensity factor; finally, adjust the embedding intensity of the watermark according to the embedding intensity factor, and output the audio frame embedded with the watermark information, so as to dynamically adjust the embedding intensity according to the characteristics of each audio, not only better protect the copyright, but also improve the anti-attack ability of the watermark without affecting the sound quality; the present application proposes a new adaptive audio robust watermark generation method based on an embedding intensity prediction network, which can be fine-tuned according to the characteristics of different audios, so that it can adapt to various types of audio content; and according to the characteristics of different audio frames, the appropriate embedding intensity is selected, so that the watermark information is better hidden without affecting the sound quality, and the imperceptibility is improved; by adaptively adjusting the embedding intensity, the robustness is improved as much as possible while ensuring the imperceptibility, so that it can resist various signal processing operations and desynchronization attacks; at the same time, the imperceptibility and robustness can be balanced, and the appropriate embedding intensity can be selected according to the specific application scenario, so as to achieve the best copyright protection effect.

[0143] Exemplary device

[0144] Based on the above embodiment, the present application further provides a terminal, comprising: a processor, a memory, an interface, a display screen and a communication module connected through a system bus; wherein the processor is used to provide computing and control capabilities; the memory includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operating system and the computer program in the storage medium to run; the interface is used to connect external devices, such as mobile terminals and computer devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0145] The computer program is executed by the processor to implement the operations of the audio robust watermark generation method based on the embedding intensity prediction network.

[0146] Those skilled in the art can understand that,Figure 6 The principle block diagram shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In an embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing an audio robust watermark generation program based on an embedding strength prediction network, the audio robust watermark generation program based on an embedding strength prediction network being used for implementing the operations of the audio robust watermark generation method based on an embedding strength prediction network as described above when executed by the processor.

[0148] In an embodiment, a computer readable storage medium is provided, wherein the computer readable storage medium stores an audio robust watermark generation program based on an embedding strength prediction network, the audio robust watermark generation program based on an embedding strength prediction network being used for implementing the operations of the audio robust watermark generation method based on an embedding strength prediction network as described above when executed by the processor.

[0149] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include non-volatile and / or volatile memory.

[0150] In summary, the present application provides an audio robust watermark generation method based on an embedding strength prediction network, a terminal and a storage medium. The method comprises: obtaining audio data, performing frame division and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix; inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor; adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame with embedded watermark information; the present application proposes a new adaptive audio robust watermark generation method based on an embedding strength prediction network, which improves the imperceptibility and robustness of the watermark algorithm.

[0151] It should be understood that the application of the present application is not limited to the above examples. Those skilled in the art can make improvements or modifications according to the above description, and all such improvements and modifications shall fall within the scope of protection of the claims of the present application.

Claims

1. An audio robust watermark generation method based on an embedding strength prediction network, characterized by, The method comprises the following steps: obtaining audio data, performing frame segmentation and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix; inputting the three-dimensional coefficient matrix into an embedding strength prediction network, and outputting a corresponding embedding strength factor; adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame embedded with watermark information; the method of obtaining audio data, performing frame segmentation and watermark embedding processing on the audio data, and generating a corresponding three-dimensional coefficient matrix comprises the following steps: obtaining the audio data, performing frame segmentation processing on the audio data, and performing short-time Fourier transform processing on each frame of audio after frame segmentation to obtain a corresponding two-dimensional coefficient matrix; splicing the real part and the imaginary part of the two-dimensional coefficient matrix, and performing watermark embedding processing to obtain the three-dimensional coefficient matrix; the method of inputting the three-dimensional coefficient matrix into an embedding strength prediction network and outputting a corresponding embedding strength factor comprises the following steps: inputting the three-dimensional coefficient matrix into the embedding strength prediction network to obtain the embedding strength factor corresponding to each frame of audio through feature splicing and mapping processing; outputting the embedding strength factor corresponding to each frame of audio through the embedding strength prediction network; the method of inputting the three-dimensional coefficient matrix into the embedding strength prediction network to obtain the embedding strength factor corresponding to each frame of audio through feature splicing and mapping processing comprises the following steps: inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix; splicing the feature representation matrix and the two-dimensional coefficient matrix in the channel, and mapping back to the initial number of channels; determining the embedding strength factor according to the mapped feature representation matrix.

2. The audio robust watermark generation method based on the embedding strength prediction network according to claim 1, characterized in that, the method of performing short-time Fourier transform processing on each frame of audio after frame segmentation to obtain a corresponding two-dimensional coefficient matrix comprises the following steps: multiplying the sample points of each frame of audio after frame segmentation by the corresponding value of the Hanning window function for weighting; obtaining the corresponding two-dimensional coefficient matrix through short-time Fourier transform of each frame of audio after windowing.

3. The audio robust watermark generation method based on the embedding strength prediction network according to claim 1, characterized in that, the method of inputting the three-dimensional coefficient matrix into a plurality of feature extraction modules to obtain a corresponding feature representation matrix comprises the following steps: inputting the three-dimensional coefficient matrix into a convolution module and four channel attention compression and excitation modules in sequence for feature enhancement to obtain a corresponding feature representation matrix.

4. The method of claim 1, wherein the embedding strength prediction network is trained using a loss function. the method of splicing the feature representation matrix and the two-dimensional coefficient matrix in the channel, and mapping back to the initial number of channels comprises the following steps: splicing the feature representation matrix and the two-dimensional coefficient matrix in the channel, and mapping back to the initial number of channels through a convolution module and a pooling module; mapping the output range of the feature representation matrix to the 0-1 interval through a sigmoid activation function.

5. The method of claim 1, wherein the embedding strength prediction network is trained using a loss function. the method of adjusting the embedding strength of the watermark according to the embedding strength factor, and outputting an audio frame embedded with watermark information comprises the following steps: adjusting the embedding strength of the watermark according to the embedding strength factor, and embedding watermark information in the audio frame according to the embedding strength of the watermark; outputting the audio frame embedded with watermark information.

6. A terminal, characterized by comprising: The method comprises the following steps: A processor and a memory, the memory storing an audio robust watermark generation program based on an embedding strength prediction network, the audio robust watermark generation program based on the embedding strength prediction network being used for implementing operations of the audio robust watermark generation method based on the embedding strength prediction network as claimed in any one of claims 1-5 when executed by the processor.

7. A computer readable storage medium characterized by The computer readable storage medium stores an audio robust watermark generation program based on an embedding strength prediction network, the audio robust watermark generation program based on the embedding strength prediction network being used for implementing operations of the audio robust watermark generation method based on the embedding strength prediction network as claimed in any one of claims 1-5 when executed by a processor.

Images