Frequency dynamic convolution based robust audio watermarking against compression encoding method and system

Abstract

The application discloses a frequency dynamic convolution-based anti-compression-encoding robust audio watermarking method and system, relates to the field of multimedia information security, and comprises the following steps: an audio frequency domain conversion step, a watermark encoding step, a watermark embedding step, an audio waveform reconstruction step and a watermark extraction step. The application improves the robustness of the watermark to various signal processing while ensuring the imperceptibility of the watermark, effectively solves the watermark failure problem caused by compression encoding in the traditional watermarking method, can better learn to embed the watermark in the robust and imperceptible area in the frequency domain, and enhances the copyright protection and traceability of the watermark in the real application scene with multiple compressions.

Description

Technical Field

[0001] This invention relates to the field of multimedia information security, and in particular to a robust audio watermarking method and system based on frequency dynamic convolution and compression-resistant coding. Background Technology

[0002] In recent years, online audio platforms have developed rapidly. However, with the large-scale circulation of audio content on online platforms, the potential risks of unauthorized copying and misuse have also increased, posing new challenges to copyright protection and content security.

[0003] Against this backdrop, robust audio watermarking technology, as an effective means of digital content protection, is widely used in audio copyright protection and source tracing. Its core idea is to embed specific watermark information, representing copyright information or identity, into the original audio in a difficult-to-remove and imperceptible manner. When source tracing is required, the watermark information can be recovered from potentially tampered or distributed audio copies using specific extraction algorithms, thus providing content creators and distribution platforms with reliable proof of ownership and evidence for infringement tracking.

[0004] From a technical perspective, robust audio watermarking technologies can currently be divided into two categories: watermarking methods based on handcrafted features and watermarking methods based on deep learning. Handcrafted feature-based audio watermarking methods embed and extract watermarks by extracting temporal features (such as temporal echoes and zero crossover rates) or commutative domain features (such as Fourier transform coefficients or discrete wavelet coefficients) of the audio signal. These methods rely on manually designed rules and are characterized by low computational complexity and ease of implementation. For example, the patchwork-based audio watermarking method based on Discrete Cosine Transform (DCT) coefficients proposed by Natgunanathan et al. (see Natgunanathan I, Xiang Y, Hua G, et al. Patchwork-based multilayer audio watermarking[J]. IEEE / ACM Transactions on Audio, Speech, and Language Processing, 2017, 25(11): 2176-2187) and the signal-to-noise ratio constrained watermarking method based on discrete wavelet transform and discrete cosine transform coefficients proposed by Su et al. (see Su Z, Zhang G, Yue F, et al. SNR-constrained heuristics for optimizing the scaling parameter of robust audiowatermarking[J]. IEEE Transactions on Multimedia, 2018, 20(10): 2631-2644). Most deep learning-based watermarking methods employ an encoder-distortion layer-decoder framework for network training. Leveraging the powerful learning capabilities of deep neural networks, they can automatically extract deeper, robust audio features and embed them into the watermark.For example, the Liu team proposed an anti-recording audio watermarking model based on a fully convolutional network (see Liu C, Zhang J, Fang H, et al. Dear: A deep-learning-based audio re-recording resilient watermarking[C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(11):13201-13209) and the Roman team proposed a location-based robust watermarking scheme based on EnCodec (see San RomanR, Fernandez P, Elsahar H, et al. Proactive Detection of Voice Cloning with Localized Watermarking[C] / / ICML 2024-41st International Conference on MachineLearning. 2024, 235: 1-17).

[0005] In practical applications, audio content often undergoes various compression and encoding processes. On the one hand, when audio is transmitted or stored on social media platforms, it may inevitably be re-encoded due to the automatic compression and transcoding strategies adopted by the platforms. On the other hand, malicious users may intentionally apply lossy compression (such as MP3 compression or Opus compression) to remove copyright watermarks or reduce their detectability when stealing and copying audio content from social media platforms. Compression encoding, in the process of reducing the bitrate, involves operations such as quantization, discarding high-frequency or low-energy signals, and frequency domain coefficient reconstruction; this process introduces quantization errors and frequency domain perturbations, causing embedded watermarks to easily become invalid or lost. Although existing work has considered robustness to compression coding, such as the DCT-based patch-style audio watermarking method proposed by Natgunanathan et al., or the FSVC watermarking method proposed by Zhao et al. (see Zhao J, Zong T, Xiang Y, et al. Desynchronization attacks resilientwatermarking method based on frequency singular value coefficient modification[J]. IEEE / ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2282-2295), these methods rely heavily on traditional manual features, making it difficult to mine deep robust features of audio. This results in insufficient robustness of watermarks under various compression coding scenarios. If copyright watermarks cannot resist compression coding, compression attacks can remove the watermark without significantly degrading the sound quality. Therefore, pirates will prioritize this method, making compression attacks a particularly prominent threat in actual infringement scenarios. Therefore, to achieve effective audio content copyright protection and traceability, there is an urgent need for an audio watermarking method that can effectively resist compression coding. Summary of the Invention

[0006] In view of the defects and shortcomings of the existing technology, the purpose of this invention is to propose a robust audio watermarking method and system based on frequency dynamic convolution, which effectively improves the robustness of the watermark to various signal processing, especially compression coding, while ensuring the imperceptibility of the watermark.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] On the one hand, a compression-resistant robust audio watermarking method based on frequency dynamic convolution includes the following steps:

[0009] The audio frequency domain transformation steps are as follows: First, the original audio signal 'a' is obtained. Then, based on the Short-Time Fourier Transform (STFT), the audio time-domain signal is converted into a frequency-domain signal, yielding the real part S of the STFT coefficients. real And the imaginary part S imag ; where the real part S real For the watermark embedding field, the imaginary part S imag Used to recover audio time-domain waveforms;

[0010] The watermarking encoding step involves encoding the watermark w to be embedded using a watermarking encoding network to obtain a high-dimensional watermark feature F that matches the shape of the watermark embedding domain. w ;

[0011] The watermark embedding step obtains the watermark feature F. w S of the Real Department real Using an embedder network to integrate watermark features F w Fusion to frequency domain S real In the middle, and output the real part S containing the watermark. w real The embedder network includes a channel dimension splicing block, several standard two-dimensional convolutional blocks, several frequency dynamic convolutional blocks, a frequency dynamic convolutional upsampling module with the same number of frequency dynamic convolutional blocks, and a standard two-dimensional convolutional upsampling module with one less than the number of standard two-dimensional convolutional blocks. The embedder network constructs multi-level feature representations in the frequency domain and watermark feature encoding stage, and uses dense skip connections to achieve feature fusion.

[0012] The audio waveform reconstruction step uses the inverse short-time Fourier transform to extract the real part S of the watermarked short-time Fourier transform coefficients. w real and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w ;

[0013] The watermark extraction step involves obtaining a segment of audio signal containing a watermark, which may have been compressed or subjected to other signal processing attacks. ’ It is input into a watermark extractor network constructed based on frequency dynamic convolution to extract watermarked audio signals a ’ Accurately extract watermarks from w ’ .

[0014] Preferably, the audio frequency domain transformation step specifically includes:

[0015] First, obtain the original audio signal a, a∈R 1×N This is divided into overlapping frames, as follows:

[0016] ;

[0017] Among them, am This represents the audio signal of the m-th frame; m is the frame index; l represents the intra-frame sampling point index; Hop represents the frame shift; Flen represents the frame length.

[0018] Then, for each frame signal a m [l] After applying the windowing function, a Discrete Fourier Transform (STFT) is performed to obtain the STFT coefficients of the frame, as follows:

[0019] ;

[0020] Where window[l] represents the window function; k represents the frequency index; S[m,k] is a complex number representing the complex STFT coefficient of the m-th frame and the k-th frequency unit;

[0021] The complex STFT coefficients are decomposed into real and imaginary parts as follows:

[0022] ;

[0023] Among them, S real ∈R H×W S represents the real part matrix of the STFT coefficients, which is the watermark embedding field; S imag ∈R H×W This represents the imaginary part matrix of the STFT coefficients, which does not carry watermark information and is only used to recover the audio time-domain waveform; j is the imaginary unit; R H×W Let H represent a real matrix of size H×W, where H and W represent the frequency dimension along the vertical direction and the number of time frames along the horizontal direction of the time-frequency representation of audio a after short-time Fourier transform, respectively.

[0024] Preferably, the watermark embedding step specifically includes:

[0025] Obtain watermark feature F w S of the Real Department real First, S real With F w By concatenating along the channel dimension, a fused feature containing both audio features and watermark information is obtained. ,as follows:

[0026] ;

[0027] in, Indicates the concatenation function; Represents a 2×H×W real matrix; H and W represent the frequency dimension along the vertical direction and the number of time frames along the horizontal direction of the time-frequency representation of audio a after short-time Fourier transform, respectively.

[0028] Subsequently, multi-layer feature fusion processing was performed. The watermark residual was obtained. ,as follows:

[0029] ;

[0030] Among them, multi-layer feature fusion processing Includes: fusion feature F c Low-level local frequency domain features are extracted using three standard 2D convolutions, with the latter two including downsampling operations. The next layer is a deep coding section containing two layers of dynamic frequency convolution blocks. These dynamic frequency convolution blocks, including downsampling operations, dynamically adjust the convolution kernel response based on the input frequency domain features, adaptively capturing and enhancing key frequency components related to the watermark information. This achieves deep fusion of audio spectral features and watermark features to obtain deep features. Subsequently, the deep features are upsampled sequentially using two dynamic frequency convolution upsampling blocks and two standard 2D convolution upsampling blocks to progressively restore spatial resolution. These multi-level upsampled features are then skipped with downsampled features of the same level.

[0031] Watermark residual With the real part S real Adding them together, we finally obtain the real part containing the watermark. ,as follows:

[0032] .

[0033] Preferably, the audio waveform reconstruction step specifically includes:

[0034] First, the real part S containing the watermark w real and the original imaginary part S imag By combining these components, we can form the complex short-time Fourier transform coefficient matrix, as follows:

[0035] ;

[0036] Among them, S w For complex frequency domain representation;

[0037] Next, the inverse discrete Fourier transform is applied to the complex frequency domain coefficients of each frame, i.e., S'[m,k], to obtain the corresponding time-domain frame signal. ,as follows:

[0038] ;

[0039] Where m represents the frame index; k represents the frequency index; y m This represents the temporal frame signal reconstructed from the m-th frame; l represents the intra-frame sampling point index; and Flen represents the frame length.

[0040] The time-domain frame signal y after inverse transformation of each frame m [l] Multiply by a window function identical to STFT. The waveforms are then overlapped and added based on their frame shift positions to obtain the complete watermarked audio time-domain waveform a. w [n], as follows:

[0041] ;

[0042] Where n represents the index of the sampling point of the entire audio waveform; Hop represents the frame shift.

[0043] Preferably, during the training of the watermarking model, Mel spectrogram loss and Mel frequency cepstral coefficient loss are introduced, and the original audio signals a and a are constrained by mean square error loss (MSE). w The similarity between them; the watermark model is an overall model including a watermark encoding network, an embedder network, and a watermark extraction network;

[0044] Mel spectrogram loss , means as follows:

[0045] ;

[0046] Where Mel represents the Mel spectrum transform function. These represent the parameters of the Mel-band spectrum converter;

[0047] Mel frequency cepstral coefficient loss , means as follows:

[0048] ;

[0049] Wherein, MFCC represents the Mel-frequency cepstral coefficient feature extraction function. This represents its parameters.

[0050] Preferably, the watermark extraction step specifically includes:

[0051] Obtain a watermarked audio signal that may have been compressed or attacked. ’ Perform a Short-Time Fourier Transform (STFT) on it to obtain its complex frequency domain matrix, which is symmetrical with the watermark embedding time. The resulting spectral signal... Decompose into real part and the virtual part ,as follows:

[0052] ;

[0053] Among them, the real part It is the main carrier field of watermark information, the imaginary part Used to ensure spectrum integrity;

[0054] Extractor network pairs Further processing is performed to recover the watermark features, as detailed below:

[0055] First, input tensors After passing through three standard 2D convolutional blocks (Conv2D), shallow local frequency domain features are extracted. The latter two convolutional blocks include downsampling operations to obtain downsampled features. ,as follows:

[0056] ;

[0057] Subsequently, features Moving to the deep coding section, this part includes two frequency-dynamic convolutional blocks (FDConv) for adaptively modeling frequency band correlations and key channel information, uncovering robust watermark features hidden deep within the audio. Each FDConv block contains downsampling operations for deep feature extraction. It is expressed as follows:

[0058] ;

[0059] Deep features After two layers of standard two-dimensional convolutional blocks that progressively restore the channel dimensions, a single-channel feature map is generated:

[0060] ;

[0061] Symmetrical to the repeated encoding operation in the watermarking stage, the feature map is subjected to mean pooling along the time dimension. Processing to obtain features ,as follows:

[0062] ;

[0063] Then the extracted features are processed through a flatten layer. Dimensional compression is performed, and the tensor shape is adjusted to fit the input of the fully connected layer, resulting in a feature tensor. It is expressed as follows:

[0064] ;

[0065] Finally, the feature tensor is processed through two fully connected Linear layers to recover the watermark vector w with the same length as the original watermark. ’ ,as follows:

[0066] .

[0067] Preferably, during the training phase of the watermark model, after the watermark embedding is completed, a distortion simulation layer is used to perform different distortion simulation operations on the watermarked audio in each training batch. Then, the watermark extractor network learns to accurately extract the watermark information from the distorted watermarked audio through an extraction loss function. Using the mean squared error loss function, it can be expressed as follows:

[0068] ;

[0069] in, This indicates the i-th bit of the watermark being extracted. This represents the i-th bit of the original embedded watermark; during the watermark model inference stage, the distortion simulation layer is discarded; the watermark model is an overall model including a watermark encoding network, an embedder network, and a watermark extraction network.

[0070] On the other hand, a compression-resistant robust audio watermarking system based on frequency dynamic convolution includes:

[0071] The audio frequency domain transformation module is used to acquire the original audio signal 'a', and convert the audio time-domain signal into a frequency-domain signal based on the Short-Time Fourier Transform (STFT), obtaining the real part S of the STFT coefficients. real And the imaginary part S imag ; where the real part S real For the watermark embedding field, the imaginary part S imag Used to recover audio time-domain waveforms;

[0072] The watermark encoding module is used to encode the watermark w to be embedded through a watermark encoding network to obtain a high-dimensional watermark feature F that is consistent with the shape of the watermark embedding domain. w ;

[0073] The watermark embedding module is used to obtain the watermark feature F. w S of the Real Department real Using an embedder network to integrate watermark features F w Fusion to frequency domain S real In the middle, and output the real part S containing the watermark. w real The embedder network includes a channel dimension splicing block, several standard two-dimensional convolutional blocks, several frequency dynamic convolutional blocks, a frequency dynamic convolutional upsampling module with the same number of frequency dynamic convolutional blocks, and a standard two-dimensional convolutional upsampling module with one less than the number of standard two-dimensional convolutional blocks. The embedder network constructs multi-level feature representations in the frequency domain and watermark feature encoding stage, and uses dense skip connections to achieve feature fusion.

[0074] The audio waveform reconstruction module is used to extract the real part S of the watermarked short-time Fourier transform coefficients using the inverse short-time Fourier transform. wreal and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w ;

[0075] The watermark extraction module is used to acquire a watermarked audio signal that may have been compressed or subjected to other signal processing attacks. ’ It is input into a watermark extractor network constructed based on frequency dynamic convolution to extract watermarked audio signals a ’ Accurately extract watermarks from w ’ .

[0076] On the other hand, an electronic device includes:

[0077] One or more processors;

[0078] Storage device for storing one or more programs.

[0079] When the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0080] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0081] (1) The embedder network of the present invention adopts a neural network structure based on frequency dynamic convolution, which realizes the deep fusion of audio frequency domain information and watermark features in the high-level feature space. The network structure can adaptively generate frequency band related convolution kernels, effectively capture the robust pattern of watermark in the spectral structure, improve the robustness of watermark to various signal processing, especially compression coding, and enable the network to better learn to embed watermark in the frequency domain in robust and imperceptible regions.

[0082] (2) This invention introduces a robust enhancement training mechanism based on distortion simulation. A distortion simulation layer is added during the watermark model training process to model common signal processing distortions. The watermark system is constrained to learn its resistance to distortion through a loss function, so as to further enhance the robustness of the watermark.

[0083] (3) The present invention performs watermark embedding processing on audio signals in the frequency domain, converts the time domain signal into a frequency domain representation through short-time Fourier transform, so that the watermark embedding acts on the spectral characteristics of the audio, and finally reconstructs the audio waveform through inverse short-time Fourier transform. This method can better control sound quality distortion and improve the imperceptibility of the watermark. Attached Figure Description

[0084] Figure 1 A flowchart of a compression-resistant, robust audio watermarking method based on frequency dynamic convolution provided in an embodiment of the present invention;

[0085] Figure 2 This is a schematic diagram of the structure of a watermarking encoding network provided in an embodiment of the present invention;

[0086] Figure 3 This is a schematic diagram of the structure of a watermark embedder network provided in an embodiment of the present invention;

[0087] Figure 4 This is a schematic diagram of the structure of a frequency dynamic convolution module provided in an embodiment of the present invention;

[0088] Figure 5 This is a schematic diagram of the structure of a watermark extractor network provided in an embodiment of the present invention;

[0089] Figure 6 This is a block diagram of a compression-resistant, robust audio watermarking system based on frequency dynamic convolution provided in an embodiment of the present invention.

[0090] Figure 7 This is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the present application. Detailed Implementation

[0091] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0092] In the description of this invention, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] like Figure 1 As shown in the figure, this invention discloses a robust audio watermarking method based on frequency dynamic convolution and compression-resistant coding. The method mainly includes five key steps: audio frequency domain transformation step S101, watermark encoding step S102, watermark embedding step S103, audio waveform reconstruction step S104, and watermark extraction step S105. The specific principles and implementation of each step are described below.

[0094] In the audio frequency domain transformation step S101, the original audio signal a is obtained, and the audio signal is transformed from a time domain signal to a frequency domain signal through a Short-Time Fourier Transform (STFT) to obtain its complex frequency domain coefficients, which are then decomposed into the real part S of the STFT coefficients. real And the imaginary part S imag The real part S real It is the embedding field of the watermark, and the imaginary part S imag Used only for audio waveform reconstruction.

[0095] The specific implementation is as follows.

[0096] First, obtain the original audio signal a, a∈R 1×N This is divided into overlapping frames, as follows:

[0097] ;

[0098] Among them, a m Let represent the audio signal of the m-th frame, l be the intra-frame sampling point index, Hop be the frame shift, and Flen be the frame length. In this embodiment, Hop is 230, Flen is 1022, the audio sampling rate is 22050 Hz, and N is the number of audio sampling points. Audio data of approximately 1 second (N = 21850) is used as the algorithm input.

[0099] Then, for each frame signal a m [l] After applying the windowing function, a Discrete Fourier Transform (STFT) is performed to obtain the STFT coefficients of the frame, as follows:

[0100] ;

[0101] Where window[l] is a window function, which is a Hamming window in this embodiment. The window length is the same as the frame length, which is 1022; k is the frequency index, and S[m,k] is a complex number representing the complex STFT coefficient of the m-th frame and the k-th frequency unit.

[0102] The complex STFT coefficients are decomposed into real and imaginary parts as follows:

[0103] ;

[0104] Among them, S real ∈R H×W It is the real part matrix of the STFT coefficients, S imag ∈R H×Wis the imaginary part matrix of the STFT coefficients, where j is the imaginary unit. H and W represent the frequency dimension (vertical direction) and time frame number (horizontal direction) of the time-frequency representation after the short-time Fourier transform of audio a, respectively. In this embodiment, the values ​​of H and W are 512 and 96, respectively. The real part S... real It is the embedding field of the watermark, with the imaginary part S. imag It does not carry watermark information and is only used to recover the audio time-domain waveform.

[0105] In the watermark encoding step S102, after obtaining the STFT coefficients of the audio, the watermark w to be embedded is encoded to obtain a high-dimensional watermark feature F that is consistent with the shape of the embedding domain. w The encoding process is achieved through methods such as... Figure 2 The watermarking coding network shown includes a fully connected layer, a non-linear activation function, and a repetitive coding operation.

[0106] The specific implementation is as follows.

[0107] For the watermark w to be embedded, w∈{0, 1} 1×L Here, L is the user-defined watermark length, which in this embodiment takes values ​​of 16, 32, 64, and 128. First, a fully connected layer is used. The initial watermark feature, mapped to a 1×H shape, is as follows:

[0108] .

[0109] Then, a non-linear activation function is used to obtain the second layer of features for the watermark. In this embodiment, the ReLU activation function is used for extraction, as follows:

[0110] ;

[0111] Last use The function repeatedly encodes the audio embedding domain along the time direction along the frame length (i.e., W) to obtain the final watermark feature, as follows:

[0112] .

[0113] Feature F w With the real part S real They are consistent in shape, which facilitates subsequent embedding.

[0114] In the watermark embedding step S103, the watermark feature F is first obtained. w The real part of the STFT of audio real Subsequently, the designed embedder network was used to integrate the watermark features F w Fusion to frequency domain S realThe embedder network combines traditional standard convolution operations with frequency dynamic convolution (FDConv) to construct multi-level feature representations in the frequency domain and watermark feature encoding stages. Dense skip connections are used to achieve feature fusion, enhancing training stability and enabling the network to learn to robustly embed the watermark into imperceptible regions in the frequency domain. The specific architecture of the embedder network is as follows... Figure 3 As shown, the network finally outputs the real part S containing the watermark. w real .

[0115] The specific implementation is as follows.

[0116] Obtain watermark feature F w S of the Real Department real First, S real With F w By concatenating functions at the channel dimension By concatenating the features, a fused feature containing both audio features and watermark information is obtained. ,as follows:

[0117] .

[0118] Subsequently, this fusion feature F c Low-level local frequency domain features are extracted using three standard two-dimensional convolutional blocks, with the latter two standard two-dimensional convolutional blocks including downsampling operations. Then, a deeper coding section containing two dynamic frequency convolutional blocks is entered, each of which includes downsampling operations. Figure 4As shown, frequency-dynamic convolution primarily achieves adaptive enhancement of input features through four processes: dynamic weight generation, fine-grained weight modulation, frequency domain feature interaction, and frequency band fusion. Specifically, firstly, average pooling, fully connected layers, and the sigmoid activation function are used to extract the global response of the input features, which is then multiplied with Fourier disjoint weights to generate dynamic convolution weights with frequency differences. Subsequently, through average pooling, fully connected layers, one-dimensional convolution, and the sigmoid activation function, modulation coefficients for modulating the convolution weights are generated from the input features. These coefficients are then modulated by linear weighted element-wise multiplication. The modulation result is then mapped to the frequency domain using a discrete Fourier transform, and frequency compression is used to filter and aggregate the frequency domain features to retain more critical frequency characteristics. Finally, these frequency features are element-wise multiplied with the input features after the discrete Fourier transform, and the result is restored to the feature space using an inverse discrete Fourier transform. Finally, the input features are processed by 2D convolution and a sigmoid activation function to generate spatial modulation coefficients, which are then multiplied element-wise with the frequency domain enhancement result. Enhanced deep embedding features are obtained through band merging. This process allows the convolution response to dynamically change with the input features, enhancing the expressive power and robustness of the watermark features in the frequency domain structure. This deep coding part outputs the enhanced deep embedding features. Subsequently, the deep embedding features are upsampled sequentially through two frequency-dynamic convolutional upsampling blocks (containing transposed convolution and frequency-dynamic convolution) and two standard 2D convolutional upsampling blocks to progressively restore spatial resolution. These multi-level upsampled features are fused with downsampled features of the same level (skip connections) to effectively alleviate the gradient vanishing problem during deep network training and enhance training stability. After the above multi-level feature fusion processing (FFP), the watermark residual is obtained. :

[0119] .

[0120] Combine it with the real part S real Adding them together, we finally obtain the real part containing the watermark. ,as follows:

[0121] .

[0122] In this way, the watermark is effectively embedded into the audio frequency domain while ensuring the imperceptibility and robustness of the embedding.

[0123] In the audio waveform reconstruction step S104, the inverse short-time Fourier transform is used to extract the real part S of the short-time Fourier transform coefficients containing the watermark. w real and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w.

[0124] The specific implementation is as follows.

[0125] First, obtain the real part S containing the watermark. w real and the original imaginary part S imag These are combined to form the complex short-time Fourier transform coefficient matrix, as follows:

[0126] .

[0127] Among them, S w The frequency domain representation is a complex number. The real part contains the embedded watermark information, and the imaginary part is used to preserve the phase information of the audio waveform, thereby ensuring the naturalness of the reconstructed audio in the time domain.

[0128] Next, for each frame's complex frequency domain coefficients, S'[m,k], the inverse discrete Fourier transform is applied to obtain the corresponding time-domain frame signal, as follows:

[0129] ;

[0130] Where m is the frame index, k is the frequency index, and y m It is the time-domain frame signal reconstructed from the m-th frame, l is the intra-frame sampling point index, and Flen is the frame length, which is 1022.

[0131] Because a window function is applied to each frame of signal during the Short Time Fourier Transform (STFT) to reduce spectral leakage, a window function is also required in the inverse transform stage. Effect compensation is performed to avoid energy distortion or amplitude distortion. To this end, the time-domain frame signal y after inverse transformation of each frame is... m [l] Multiply by a window function identical to the STFT (such as a Hamming window), and perform overlapping addition based on its frame shift position to finally obtain the complete watermarked audio time-domain waveform a. w [n], as follows:

[0132] ;

[0133] Where n is the sampling point index of the entire audio waveform, and Hop is the frame shift, which is 2^30 in this embodiment. Through this operation, the energy between overlapping frames is evenly compensated, avoiding amplitude distortion or audio energy distortion, while ensuring the smoothness and naturalness of the reconstructed audio. Finally, the obtained a... w This is the complete watermarked time-domain audio.

[0134] To ensure better speech quality for watermarked audio, the watermarking model (including the overall model of the watermark encoding network, embedder network, and watermark extraction network) incorporates Mel spectrogram loss and MFCC feature loss during training. The human ear's perception of frequencies in different audio frequencies is not linearly distributed, but it is more sensitive to the mid-to-low frequency region. Mel spectrograms resample the audio spectrum using the Mel scale, which better matches the characteristics of human hearing. Therefore, it can effectively measure the auditory similarity between the watermarked audio and the original audio in terms of human auditory perception. Mean Squared Error (MSE) is used to constrain their similarity. The Mel spectrogram loss is expressed as follows:

[0135] ;

[0136] Where Mel represents the Mel spectrum transform function. The parameters of the Mel spectrum converter are: sampling rate 22050, frame length and window length 1024, frame shift 256, maximum cutoff frequency 8000 Hz, and Hamming window.

[0137] Simultaneously, an MSE loss based on Mel-frequency cepstral coefficients (MFCC) is introduced, as follows:

[0138] ;

[0139] Where MFCC represents the MFCC feature extraction function, The MFCC feature dimension is 32 in this embodiment, representing its parameters. This step ensures the speech perception quality of the audio after watermark embedding through strict frequency-domain to time-domain mapping, window function compensation, and perceptual loss constraint training.

[0140] In the watermark extraction step S105, a segment of audio signal a containing a watermark, which may have been compressed or subjected to other signal processing attacks, is first acquired. ’ The watermark is then fed into a watermark extractor network built based on frequency dynamic convolution to accurately extract the watermark w from the distorted audio. ’ . like Figure 5 As shown, this extractor network mines deep watermark features that remain stable under attacks such as noise and compression through multi-layer frequency dynamic convolution. Subsequently, the back-end of the network maps the high-dimensional robust features into the final watermark bit sequence through a flat layer and two fully connected layers, achieving stable and reliable watermark extraction.

[0141] The specific implementation is as follows.

[0142] Obtain a watermarked audio signal that may have been compressed or attacked. ’Perform a short-time Fourier transform (STFT) on it to obtain its complex frequency domain matrix. Decompose the resulting spectral signal into its real part S. ’ real And the imaginary part S ’ imag ,as follows:

[0143] ;

[0144] Among them, the real part S ’ real It is the main domain carrying watermark information, the imaginary part S ’ imag Used to ensure spectral integrity. The extractor network primarily processes S... ’ real From this, we can learn and recover the watermark features.

[0145] See Figure 5 As shown, first, input tensor S ’ real The process involves three standard 2D convolutional blocks (Conv2D), with the latter two blocks including downsampling operations to extract shallow local frequency domain features. :

[0146] ;

[0147] Subsequently, this feature is incorporated into the deep encoding section, which includes two layers of frequency dynamic convolutional blocks (FDConv). Each FDConv block contains a downsampling operation to adaptively model frequency band correlations and key channel information, thereby uncovering robust watermark features hidden deep within the audio. It is expressed as follows:

[0148] ;

[0149] After deep features are processed through two layers of standard two-dimensional convolutional blocks that progressively restore the channel dimensions, a single-channel feature map is generated. ,as follows:

[0150] .

[0151] Symmetrical to the repeated encoding operation in the watermarking stage, the feature map is subjected to mean pooling along the time dimension (horizontal direction) to obtain the mean pooled feature map. ,as follows:

[0152] ;

[0153] Then the extracted features are processed through a flatten layer. Dimensional compression is performed to obtain Adjust the tensor shape to fit the fully connected layer input, as follows:

[0154] ;

[0155] Finally, the features are processed through two fully connected (Linear) layers to recover a watermark vector with the same length as the original watermark. ,as follows:

[0156] .

[0157] During the watermark model training phase, after the watermark is embedded, each batch of watermarked audio undergoes different distortion simulation operations (such as Gaussian noise, reweighting, MP3 compression encoding, Opus compression encoding, etc.). Then, the watermark extractor network learns to accurately extract the watermark information from the distorted watermarked audio using an extraction loss function. The extraction loss function uses the mean squared error loss function. , means as follows:

[0158] .

[0159] Among them, w ’ i w represents the i-th bit of the watermark being extracted. i This represents the i-th bit of the original embedded watermark. During the watermark model inference phase, the distortion layer is discarded. This robust enhancement training strategy based on distortion simulation helps to further improve the robustness of the watermark.

[0160] In summary, the proposed frequency-dynamic convolution-based robust audio watermarking method improves the watermark's robustness to various signal processing methods while ensuring its imperceptibility. It effectively solves the problem of watermark failure caused by compression coding in traditional watermarking methods, and enhances the watermark's copyright protection and traceability capabilities in real-world application scenarios with multiple compressions.

[0161] like Figure 6 As shown, this embodiment also discloses a compression-resistant robust audio watermarking system based on frequency dynamic convolution, including:

[0162] The audio frequency domain transformation module 601 is used to acquire the original audio signal a, and convert the audio time-domain signal into a frequency-domain signal based on the short-time Fourier transform (STFT) to obtain the real part S of the short-time Fourier transform coefficients. real And the imaginary part S imag ; where the real part S real For the watermark embedding field, the imaginary part S imag Used to recover audio time-domain waveforms;

[0163] Watermark encoding module 602 is used to encode the watermark w to be embedded through a watermark encoding network to obtain a high-dimensional watermark feature F that is consistent with the shape of the watermark embedding domain. w ;

[0164] Watermark embedding module 603 is used to obtain watermark features F w S of the Real Department real Using an embedder network to integrate watermark features F w Fusion to frequency domain S real In the middle, and output the real part S containing the watermark. w real The embedder network includes a channel dimension splicing block, several standard two-dimensional convolutional blocks, several frequency dynamic convolutional blocks, a frequency dynamic convolutional upsampling module with the same number of frequency dynamic convolutional blocks, and a standard two-dimensional convolutional upsampling module with one less than the number of standard two-dimensional convolutional blocks. The embedder network constructs multi-level feature representations in the frequency domain and watermark feature encoding stage, and uses dense skip connections to achieve feature fusion.

[0165] Audio waveform reconstruction module 604 is used to extract the real part S of the watermarked short-time Fourier transform coefficients using inverse short-time Fourier transform. w real and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w ;

[0166] Watermark extraction module 605 is used to acquire a watermarked audio signal a that may have been compressed or subjected to other signal processing attacks. ’ It is input into a watermark extractor network constructed based on frequency dynamic convolution to extract watermarked audio signals a ’ Accurately extract watermarks from w ’ .

[0167] The specific implementation process of a frequency-dynamic convolution-based compression-resistant robust audio watermarking system is basically the same as the above-described method embodiments, and will not be repeated here. It should be noted that each functional module in the system corresponds to a specific step in the method embodiments, and those skilled in the art can understand the specific implementation of the system based on the method flow.

[0168] like Figure 7 As shown, it illustrates a structural schematic diagram of a computer device 700 suitable for implementing electronic devices according to embodiments of the present application. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0169] like Figure 7As shown, the computer device 700 includes a central processing unit (CPU) 701 and a graphics processing unit (GPU) 702, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 703 or programs loaded from storage section 709 into random access memory (RAM) 704. The RAM 704 also stores various programs and data required for the operation of the device 700. The CPU 701, GPU 702, ROM 703, and RAM 704 are interconnected via bus 705. An input / output (I / O) interface 706 is also connected to bus 705. The following components are connected to the I / O interface 706: an input section 707 including a keyboard, mouse, etc.; an output section 708 including an LCD, speakers, etc.; a storage section 709 including a hard disk, etc.; and a communication section 710 including a network interface card, such as a LAN card or modem. The communication section 710 performs communication processing via a network such as the Internet. A driver 711 may also be connected to the I / O interface 706 as needed. Removable media 712, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on drive 711 as needed so that computer programs read from them can be installed into storage section 709 as needed.

[0170] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including instructions for performing the processes. Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 710, and / or installed from removable medium 712. When the computer program is executed by central processing unit (CPU) 701 and graphics processing unit (GPU) 702, it performs the functions defined in the method of this application.

[0171] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than a computer-readable medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device, apparatus, or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0172] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based means to perform the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0174] The modules described in the embodiments of this application can be implemented in software or hardware. These modules can also be located within a processor.

[0175] The above embodiments illustrate the basic principles and implementation methods of the present invention, aiming to help understand the core concept and key steps of the invention. It should be understood that these embodiments are merely examples and do not limit the scope of application of the present invention. Those skilled in the art, based on their understanding of the concept of the present invention, can make various equivalent improvements to specific steps, parameter configurations, or system structures. These improvements also fall within the protection scope of the present invention, as defined in the appended claims.

Claims

1. A robust audio watermarking method based on frequency dynamic convolution, characterized in that, include: The audio frequency domain transformation steps are as follows: First, the original audio signal 'a' is obtained. Then, based on the Short-Time Fourier Transform (STFT), the audio time-domain signal is converted into a frequency-domain signal, yielding the real part S of the STFT coefficients. real And the imaginary part S imag ; where the real part S real For the watermark embedding field, the imaginary part S imag Used to recover audio time-domain waveforms; The watermarking encoding step involves encoding the watermark w to be embedded using a watermarking encoding network to obtain a high-dimensional watermark feature F that matches the shape of the watermark embedding domain. w ; The watermark embedding step obtains the watermark feature F. w S of the Real Department real Using an embedder network to integrate watermark features F w Fusion to frequency domain S real The fusion characteristics are obtained in the middle , for fusion features Perform multi-layer feature fusion processing The watermark residual was obtained. Watermark Residue With the real part S real Adding them together, we get the real part S containing the watermark. w real The output is as follows: The embedder network includes sequentially connected channel-dimensional concatenation blocks, several standard two-dimensional convolutional blocks, several frequency-dynamic convolutional blocks, a frequency-dynamic convolutional upsampling module with the same number of frequency-dynamic convolutional blocks, and a standard two-dimensional convolutional upsampling module with one less than the number of standard two-dimensional convolutional blocks. The embedder network constructs multi-level feature representations in the frequency domain and watermark feature encoding stage, and uses dense skip connections to achieve feature fusion; wherein, multi-level feature fusion processing... Includes: fusing features F c Low-level local frequency domain features are extracted using three standard 2D convolutions, with the latter two including downsampling operations. The next layer is a deep coding section containing two layers of dynamic frequency convolution blocks. These dynamic frequency convolution blocks, including downsampling operations, dynamically adjust the convolution kernel response based on the input frequency domain features, adaptively capturing and enhancing key frequency components related to the watermark information. This achieves deep fusion of audio spectral features and watermark features to obtain deep features. Subsequently, the deep features are upsampled sequentially using two dynamic frequency convolution upsampling blocks and two standard 2D convolution upsampling blocks to progressively restore spatial resolution. These multi-level upsampled features are then skipped with downsampled features of the same level. The audio waveform reconstruction step uses the inverse short-time Fourier transform to extract the real part S of the watermarked short-time Fourier transform coefficients. w real and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w ; The watermark extraction step involves obtaining a segment of audio signal containing a watermark, which may have been compressed or subjected to other signal processing attacks. ’ It is input into a watermark extractor network constructed based on frequency dynamic convolution to extract watermarked audio signals a ’ Accurately extract watermarks from w ’ .

2. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 1, characterized in that, The audio frequency domain transformation step specifically includes: First, obtain the original audio signal a, a∈R 1×N This is divided into overlapping frames, as follows: ； Among them, a m This represents the audio signal of the m-th frame; m is the frame index; l represents the intra-frame sampling point index; Hop represents the frame shift; Flen represents the frame length. Then, for each frame signal a m [l] After applying the windowing function, a Discrete Fourier Transform (STFT) is performed to obtain the STFT coefficients of the frame, as follows: ； Where window[l] represents the window function; k represents the frequency index; S[m,k] is a complex number representing the complex STFT coefficient of the m-th frame and the k-th frequency unit; The complex STFT coefficients are decomposed into real and imaginary parts as follows: ； Among them, S real ∈R H×W S represents the real part matrix of the STFT coefficients, which is the watermark embedding field; S imag ∈R H×W This represents the imaginary part matrix of the STFT coefficients, which does not carry watermark information and is only used to recover the audio time-domain waveform; j is the imaginary unit; R H×W Let H represent a real matrix of size H×W, where H and W represent the frequency dimension along the vertical direction and the number of time frames along the horizontal direction of the time-frequency representation of audio a after short-time Fourier transform, respectively.

3. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 1, characterized in that, The watermark encoding step specifically includes: For the watermark to be embedded w∈{0, 1} 1×L First, use a fully connected layer. The first layer of features maps the watermark into a 1×H shape. ,as follows: ； in, Represents a 1×H real matrix; Then a nonlinear activation function is used. Obtain the second layer of features of the watermark ,as follows: ； Finally, based on the repeated encoding of W, the final watermark feature is obtained. ,as follows: ； in, Indicates a repeating encoding function; Represents a 1×H×W real matrix; characteristic F w With the real part S real They are consistent in shape; H and W represent the frequency dimension along the vertical direction and the number of time frames along the horizontal direction of the time-frequency representation of audio a after short-time Fourier transform, respectively.

4. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 1, characterized in that, The watermark embedding step specifically includes: Obtain watermark feature F w S of the Real Department real First, S real With F w By concatenating along the channel dimension, a fused feature containing both audio features and watermark information is obtained. ,as follows: ； in, Indicates the concatenation function; Represents a 2×H×W real matrix; H and W represent the frequency dimension along the vertical direction and the number of time frames along the horizontal direction of the time-frequency representation of audio a after short-time Fourier transform, respectively. Subsequently, multi-layer feature fusion processing was performed. The watermark residual was obtained. ,as follows: ； Watermark residual With the real part S real Adding them together, we finally obtain the real part containing the watermark. ,as follows: 。 5. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 1, characterized in that, The audio waveform reconstruction step specifically includes: First, the real part S containing the watermark w real and the original imaginary part S imag By combining these components, we can form the complex short-time Fourier transform coefficient matrix, as follows: ； Among them, S w For complex frequency domain representation; Next, the inverse discrete Fourier transform is applied to the complex frequency domain coefficients of each frame, i.e., S'[m,k], to obtain the corresponding time-domain frame signal. ,as follows: ； Where m represents the frame index; k represents the frequency index; y m This represents the temporal frame signal reconstructed from the m-th frame; l represents the intra-frame sampling point index; Flen represents the frame length; The time-domain frame signal y after inverse transformation of each frame m [l] Multiply by a window function identical to STFT. The waveforms are then overlapped and added based on their frame shift positions to obtain the complete watermarked audio time-domain waveform a. w [n], as follows: ； Where n represents the index of the sampling point of the entire audio waveform; Hop represents the frame shift.

6. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 5, characterized in that, During the training of the watermarking model, Mel spectrogram loss and Mel frequency cepstral coefficient loss are introduced, and the original audio signals a and a are constrained by mean square error loss (MSE). w The similarity between them; the watermark model is an overall model including a watermark encoding network, an embedder network, and a watermark extractor network; Mel spectrogram loss , means as follows: ； Where Mel represents the Mel spectrum transform function. These represent the parameters of the Mel-band spectrum converter; Mel frequency cepstral coefficient loss , means as follows: ； Wherein, MFCC represents the Mel-frequency cepstral coefficient feature extraction function. This represents its parameters.

7. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 1, characterized in that, The watermark extraction step specifically includes: Obtain a watermarked audio signal that may have been compressed or attacked. ’ Perform a Short-Time Fourier Transform (STFT) on it to obtain its complex frequency domain matrix, which is symmetrical with the watermark embedding time. The resulting spectral signal... Decompose into real part and the virtual part ,as follows: ； Among them, the real part It is the main carrier field of watermark information, the imaginary part Used to ensure spectrum integrity; Extractor network pairs Further processing is performed to recover the watermark features, as detailed below: First, input tensors After passing through three standard 2D convolutional blocks (Conv2D), shallow local frequency domain features are extracted. The latter two convolutional blocks include downsampling operations to obtain downsampled features. ,as follows: ； Subsequently, features Moving to the deep coding section, this part includes two frequency-dynamic convolutional blocks (FDConv) for adaptively modeling frequency band correlations and key channel information, uncovering robust watermark features hidden deep within the audio. Each FDConv block contains downsampling operations for deep feature extraction. It is expressed as follows: ； Deep features After two layers of standard two-dimensional convolutional blocks that progressively restore the channel dimensions, a single-channel feature map is generated: ； Symmetrical to the repeated encoding operation in the watermarking stage, the feature map is subjected to mean pooling along the time dimension. Processing to obtain features ,as follows: ； Then the extracted features are processed through a flatten layer. Dimensional compression is performed, and the tensor shape is adjusted to fit the input of the fully connected layer, resulting in a feature tensor. It is expressed as follows: ； Finally, the feature tensor is processed through two fully connected Linear layers to recover the watermark vector w with the same length as the original watermark. ’ ,as follows: 。 8. The compression-resistant robust audio watermarking method based on frequency dynamic convolution according to claim 7, characterized in that, During the training phase of the watermark model, after the watermark embedding is completed, a distortion simulation layer is used to perform different distortion simulation operations on each batch of watermarked audio. Then, the watermark extractor network learns to accurately extract watermark information from the distorted watermarked audio through an extraction loss function. Using the mean squared error loss function, it can be expressed as follows: ； in, This indicates the i-th bit of the watermark being extracted. This represents the i-th bit of the original embedded watermark; during the watermark model inference stage, the distortion simulation layer is discarded; the watermark model is an overall model including a watermark encoding network, an embedder network, and a watermark extractor network.

9. A robust audio watermarking system based on frequency dynamic convolution, characterized in that, The system for implementing the method according to any one of claims 1 to 8 comprises: The audio frequency domain transformation module is used to acquire the original audio signal 'a', and convert the audio time-domain signal into a frequency-domain signal based on the Short-Time Fourier Transform (STFT), obtaining the real part S of the STFT coefficients. real And the imaginary part S imag ; where the real part S real For the watermark embedding field, the imaginary part S imag Used to recover audio time-domain waveforms; The watermark encoding module is used to encode the watermark w to be embedded through a watermark encoding network to obtain a high-dimensional watermark feature F that is consistent with the shape of the watermark embedding domain. w ; The watermark embedding module is used to obtain the watermark feature F. w S of the Real Department real Using an embedder network to integrate watermark features F w Fusion to frequency domain S real In the middle, and output the real part S containing the watermark. w real The embedder network includes a channel dimension splicing block, several standard two-dimensional convolutional blocks, several frequency dynamic convolutional blocks, a frequency dynamic convolutional upsampling module with the same number of frequency dynamic convolutional blocks, and a standard two-dimensional convolutional upsampling module with one less than the number of standard two-dimensional convolutional blocks. The embedder network constructs multi-level feature representations in the frequency domain and watermark feature encoding stage, and uses dense skip connections to achieve feature fusion. The audio waveform reconstruction module is used to extract the real part S of the watermarked short-time Fourier transform coefficients using the inverse short-time Fourier transform. w real and the original imaginary part S imag The time-domain waveform is reconstructed to obtain the watermarked audio a. w ; The watermark extraction module is used to acquire a watermarked audio signal that may have been compressed or subjected to other signal processing attacks. ’ It is input into a watermark extractor network constructed based on frequency dynamic convolution to extract watermarked audio signals a ’ Accurately extract watermarks from w ’ .

10. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

Images