Systems and methods for audio watermarking

Abstract

Described is a neural network based system for use in audio watermarking, comprising: a neural network based watermark generator configured for embedding, based on a learnable embedding table, a predefined watermark into an input audio signal, thereby obtaining a watermarked audio signal; and a neural network based watermark detector comprising: a detection head that is configured for determining presence or absence of a watermark in a target audio signal; and a decoding head that is configured for decoding the watermark from the target audio signal, wherein the embedding table is shared between the watermark generator and the watermark detector, such that the decoding of the watermark from the target audio signal is based on the embedding table that has been used for the embedding.

Description

[0001] SYSTEMS AND METHODS FOR AUDIO WATERMARKING CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority from U. S. Provisional Application Ser. No. 63 / 720,675, filed on 14 November 2024, and U. S. Provisional Application Ser. No.

[0003] 63 / 886,014, filed on 22 September 2025, each of which is incorporated by reference herein in its entirety.

[0004] TECHNICAL FIELD

[0005] The present disclosure generally relates to the field of audio processing. In particular, the present disclosure relates to systems and methods for audio watermarking (e.g., embedding, detection, and / or decoding).

[0006] BACKGROUND

[0007] Recent advancements in audio synthesis technologies, particularly in zero- shot voice synthesis, have raised significant concerns regarding voice fraud and speaker impersonation. To counter these risks, various audio watermarking methods have been proposed. These watermarking methods are crucial in hiding information in audio signals for authentication, attribution, and copyright protection. However, existing audio watermarking techniques may appear to face significant challenges in learning efficiency, attribution accuracy, and robustness against modern audio transformations.

[0008] In particular, one issue is that these methods are primarily designed to be robust against traditional audio preprocessing transformations, such as MP3 compression and band-based filtering. However, these methods are not equipped to handle Al-based editing transformations that leverage advanced Al models, including audio diffusion models and language models, to edit audio. These Al-based transformations can significantly alter the audio signal in ways that traditional watermarking approaches are not robust against.

[0009] Moreover, existing methods appear to train the watermarking and detection models separately without shared internal components, such as the embedding tables of the message encoder. This separation forces the detection model to reverse-engineer the watermarking process independently, resulting in slower learning rates and reduced attribution accuracy. Additionally, simplistic techniques for composing hidden message states, such as direct averaging, fail to capture complex patterns in the data, leading to suboptimal performance. Thus, there is a need for methods and systems for use in audio watermarking (e.g., embedding, detection, and / or decoding), and possibly also for methods for enabling training of such systems for use in watermarking (embedding, detection, and / or decoding) with improved performance and / or efficiency.

[0010] SUMMARY

[0011] In view of the above, the present disclosure generally provides a neural network (NN) based system for use in audio watermarking, a method of training a neural network based system for use in audio watermarking, a method of a neural network based system for audio watermarking, a method of a watermark detector for audio watermark detection and decoding, a watermark detector, as well as a corresponding apparatus, a program, and a computer-readable storage medium, having the features of the respective independent claims. The dependent claims relate to preferred embodiments.

[0012] According to a first aspect of the disclosure, there is provided a neural network based system for use in audio (e.g., speech, music, etc.) watermarking (e.g., embedding, detection, and / or decoding, or the like). The system may comprise a neural network based watermark generator configured for embedding, based on a learnable embedding table, a predefined watermark into an input audio signal, thereby obtaining a watermarked audio signal. The embedding table may be learned during the course of the training of such a system, for example. The system may further comprise a neural network based watermark detector. The watermark detector itself may comprise: a (watermark) detection head that is configured for determining presence or absence of a watermark in a target audio signal; and a (watermark) decoding head that is configured for decoding the watermark from the target audio signal. More particularly, the embedding table may be shared between the watermark generator and the watermark detector, such that the decoding of the watermark from the target audio signal may be based on the embedding table that has been used for the embedding.

[0013] Configured as proposed, the present disclosure generally provides a (neural network based) system for use in audio watermarking processes which achieves improved performance in terms of efficiency, robustness against wide range of audio transformations, etc., in both (watermark) detection and attribution, while at the same time also preserving perceptual quality.

[0014] In some example embodiments, the watermark comprises a binary message sequence. For instance, depending on various implementations, the watermark may comprise a predefined binary message sequence, or the like. In some possible cases (e.g., during training), the watermark may comprise a randomly generated binary sequence of a predefined length (e.g., K-bit).

[0015] In some example embodiments, the decoding head may comprise a cross-attention module configured for decoding the watermark based on the shared embedding table. As will become more apparent with the detailed description below, by utilizing the shared embedding table as a bridge together with the cross-attention module, it generally allows for more efficient learning and accurate (watermark) message retrieval.

[0016] In some example embodiments, the watermark generator may comprise an encoder module configured for transforming the input audio signal into a latent representation for the embedding.

[0017] In some example embodiments, the watermark generator may further comprise a message encoder module (or sometimes also referred to as a message / watermark embedder) configured for mapping the watermark to its latent representation, so as to enable embedding of the latent representation of the watermark into the latent representation of the input audio signal in the latent space, thereby obtaining a modulated latent representation of the input audio signal.

[0018] In some example embodiments, the latent representation of the watermark may be obtained based on a temporal message conditioning mechanism that employs a linear modulation layer. As will become more apparent with the detailed description below, this simple approach not only prevents the sole reliance on the frequency domain for distributing the K-bit (watermark) message (as in some of the existing techniques), but also significantly facilitates the learning process for message decoding.

[0019] In some example embodiments, the watermark generator may further comprise a decoder module configured for decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained. In particular, the decoder module may share a symmetric structure with the encoder module. In some example embodiments, the watermark detector may also comprise an encoder module (analogous or similar to that of the watermark generator) configured for transforming the target audio signal into a reconstructed latent representation.

[0020] In some example embodiments, the cross attention module may be configured for decoding the watermark based on the reconstructed latent representation, by referencing the embedding table (or, in other words, by using the shared embedding table as a reference).

[0021] In some example embodiments, the embedding table (e.g., after being shared with the watermark detector) may be transformed into a key-value based reference based on linear projection for being used in the referencing.

[0022] In some example embodiments, the decoding of the watermark may further involve attention-based vector merging with a query from the reconstructed latent representation.

[0023] In some example embodiments, the neural network based system may have been trained end-to-end based on at least one loss function.

[0024] In some example embodiments, the at least one loss function may comprise a perceptual loss function.

[0025] In some example embodiments, the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency (or time-frequency, TF) masking loss based on a psychoacoustic masking principle. Of course, as will be understood and appreciated by the skilled person, any other suitable loss function (perceptual or not) may be adopted as well, e.g., alone or in combination, depending on various implementations and / or requirements.

[0026] In some example embodiments, the watermark detector may be further configured to perform watermark localization based on a sliding window detection mechanism. For instance, in some possible implementations, assuming the audio signal is represented as a sequence of frames, the proposed localization approach may utilize multiple overlapping detection windows, where each window may correspond to an audio segment (e.g., of 1 second). For each window, a (per-window) watermark detection probability may be computed, indicative of the presence (or the absence) of the watermark. As a given frame may be part of several such overlapping windows, the per- frame detection probability may then be determined by aggregating the probabilities from all windows covering that frame. This localization process may be considered to effectively distribute the segment-level detection scores to a finer, perframe resolution, enabling precise localization of the watermarked regions. According to a second aspect of the disclosure, there is provided a method of training a neural network based system for use in audio watermarking. The neural network based system may be implemented according to the preceding aspect, or may be implemented in any other suitable means. For instance, the neural network based system may comprise: a neural network based watermark generator for embedding a watermark into an input audio signal; and a neural network based watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal. More particularly, the method may comprise obtaining, as a training input (or in other words, input(s) for the training), a test audio signal and a test watermark sequence that are used as ground truth. The method may further comprise embedding, by using the watermark generator and based on a learnable embedding table, the test watermark sequence into the test audio signal, thereby obtaining a watermarked audio signal. The method may yet further comprise predicting, based on the watermarked audio signal, presence of a watermark by using the detection head, and an estimated watermark sequence by using the decoding head. In addition, the method may comprise iteratively training the neural network based system based on at least one loss function indicative of respective difference between the predicted presence, the estimated watermark sequence, and the ground truth. More particularly, the method may further comprise sharing the embedding table between the watermark generator and the watermark detector, such that the prediction of the estimated watermark sequence may be based on the embedding table that has been used for the embedding. In this sense, such process may also be referred to as shared learning. That is, the decoding (prediction) of the watermark sequence may be simply understood to reuse the embedding table that has been learned. Configured as proposed, the present disclosure generally provides an efficient, flexible yet reliable mechanism of training a (neural network based) system for use in audio watermarking processes (e.g., embedding, detection, decoding, etc.) which achieves improved performance in terms of efficiency, robustness against wide range of audio transformations, etc., in both (watermark) detection and attribution, while at the same time also preserving perceptual quality.

[0027] In some example embodiments, the decoding head may comprise a cross-attention module. In particular, the estimated watermark sequence may be predicted by the cross-attention module based on the shared embedding table. As will become more apparent with the detailed description below, by utilizing the shared embedding table as a bridge together with the cross-attention module, it generally allows for more efficient learning and accurate (watermark) message retrieval.

[0028] In some example embodiments, the watermark generator may comprise an encoder module. In particular, the method may further comprise, before the embedding, transforming, by using the encoder module, the test audio signal into a latent representation.

[0029] In some example embodiments, the watermark generator may further comprise a message encoder module (or sometimes also referred to as a message / watermark embedder).

[0030] Accordingly, the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal may involve mapping, by using the message encoder module, the test watermark sequence into its latent representation, so as to enable embedding of the latent representation of the test watermark sequence into the latent representation of the test audio signal in the latent space, thereby obtaining a modulated latent representation of the test audio signal.

[0031] In some example embodiments, the latent representation of the test watermark sequence may be obtained based on a temporal message conditioning mechanism that employs a linear modulation layer. As will become more apparent with the detailed description below, this simple approach not only prevents the sole reliance on the frequency domain for distributing the K-bit (watermark) message (as in some of the existing techniques), but also significantly facilitates the learning process for message decoding.

[0032] In some example embodiments, the watermark generator may further comprise a decoder module that shares a symmetric structure with the encoder module. Accordingly, the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal, in order to obtain a watermarked audio signal may further comprise decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained.

[0033] In some example embodiments, the watermark detector may also comprise an encoder module (analogous or similar to that of the watermark generator). Accordingly, the method may further comprise, before the predicting, transforming, by using the encoder module, the watermarked audio signal into a reconstructed latent representation. In some example embodiments, the estimated watermark sequence may be predicted by the cross attention module based on the reconstructed latent representation, by referencing the embedding table (or, in other words, by using the shared embedding table as a reference). In some example embodiments, the method may further comprise, before the prediction of the estimated watermark sequence, transforming the embedding table into a key- value based reference based on linear projection.

[0034] In some example embodiments, the prediction of the estimated watermark sequence may further involve attention-based vector merging with a query from the reconstructed latent representation.

[0035] In some example embodiments, the at least one loss function may comprise a perceptual loss function.

[0036] In some example embodiments, the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency (TF) masking loss based on a psychoacoustic masking principle. Of course, as will be understood and appreciated by the skilled person, any other suitable loss function (perceptual or not) may be adopted as well, e.g., alone or in combination, depending on various implementations and / or requirements.

[0037] In some example embodiments, the psychoacoustic-aligned TF masking loss may involve a weighted L2 loss in the Mel-spectrogram space, using a per-tile weight with asymmetric TF decay.

[0038] In some example embodiments, the psychoacoustic-aligned TF masking loss may involve: identifying at least one masker region; applying TF decay around the masker region, so as to determine a masking threshold; determining at least one maskee region based on the masking threshold; and adjusting weights of the psychoacoustic-aligned TF masking loss, so as to encourage more watermark signal to be embedded in the maskee region.

[0039] In some example embodiments, the method may further comprise, before the predicting, applying at least one audio transformation to the watermarked audio signal to obtain a transformed audio signal; and feeding the transformed audio signal into the watermark detector. The audio transformation may be applied in the waveform domain, for example. In some example embodiments, the audio transformation may comprise at least one of: compression, cropping, filtering, boost / duck, echo, noise addition, or artificial intelligence (Al) based technique. Of course, as will be understood and appreciated by the skilled person, any other suitable transformation may be adopted as well, alone or in combination, depending on various implementations and / or requirements.

[0040] In some example embodiments, applying the Al based transformation may be based on a set of principal components in noise space obtained by principal component analysis (PCA). In some example embodiments, the iteratively training of the neural network based system may comprise updating parameters (e.g., weights, or the like) of the watermark generator and / or the watermark detector based on evaluation of the at least one loss function.

[0041] According to a third aspect of the disclosure, there is provided a method of a neural network based system for audio watermarking. The neural network based system may be implemented according to the first aspect, or may be implemented in any other suitable means. For instance, the neural network based system may comprise: a (neural network based) watermark generator for embedding a watermark into an input audio signal based on a learnable embedding table; and a (neural network based) watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal. In particular, the embedding table is shared between the watermark generator and the watermark detector. More particularly, the method may comprise: obtaining the input audio signal; obtaining the watermark; and embedding, by using the watermark generator and based on the embedding table, the watermark into the input audio signal, thereby obtaining a watermarked audio signal.

[0042] In some example embodiments, the method may further comprise: obtaining the target audio signal; determining, by using the detection head, presence or absence of a watermark in the target audio signal; and decoding the watermark from the target audio signal by using the decoding head based on the embedding table.

[0043] According to a fourth aspect of the disclosure, there is provided a method of a watermark detector for audio watermark detection and decoding. In particular, the watermark detector may comprise a detection head and a decoding head. More particularly, the method may comprises: obtaining a target audio signal; determining, by using the detection head, presence or absence of a watermark in the target audio signal; and decoding, by using the decoding head, the watermark from the target audio signal based on a learnable embedding table, wherein the embedding table has been used by a watermark generator during training for embedding a watermark into an input audio signal.

[0044] According to a fifth aspect of the disclosure, there is provided a watermark detector. In particular, the watermark detector may comprise a detection head configured for determining presence or absence of a watermark in a target audio signal. The watermark detector may further comprise a decoding head configured for decoding the watermark from the target audio signal based on a learnable embedding table. More particularly, the embedding table may have been used by a watermark generator during training for embedding a watermark into an input audio signal.

[0045] According to a fifth aspect of the disclosure, there is provided an apparatus. The apparatus may comprise a processor and a memory coupled to the processor. In particular, the processor may be adapted to carry out any of the example methods described in the foregoing aspects. According to a sixth aspect of the present disclosure, there is provided a computer program. The computer program may include instructions that, when executed by a processor, cause the processor to carry out all steps of the example methods described throughout the present disclosure.

[0046] According to a seventh aspect of the present disclosure, there is provided a computer-readable storage medium. The computer-readable storage medium may store the aforementioned computer program.

[0047] It will be appreciated that apparatus features and method steps may be interchanged in many ways. In particular, the details of the disclosed method(s) can be realized by the corresponding apparatus (or system), and vice versa, as the skilled person will appreciate. Moreover, any of the above statements made with respect to the method(s) are understood to likewise apply to the corresponding apparatus (or system), and vice versa.

[0048] BRIEF DESCRIPTION OF DRAWINGS

[0049] Example embodiments of the disclosure are explained below with reference to the accompanying drawings, wherein

[0050] Fig. 1 is a schematic illustration of an example block diagram of a neural network based system for use in audio watermarking according to some example embodiments of the present disclosure,

[0051] Fig. 2 is a flowchart illustrating an example of a method of training a neural network based system for use in audio watermarking according to some example embodiments of the present disclosure,

[0052] Fig. 3 is a flowchart illustrating an example of a method of a neural network based system for audio watermarking according to some example embodiments of the present disclosure, Fig. 4 is a flowchart illustrating an example of a method of a watermark detector for audio watermark detection and decoding according to some example embodiments of the present disclosure, and

[0053] Figs. 5 to 17 are example illustrations showing various results and comparisons based on the example embodiments of the disclosure.

[0054] DETAILED DESCRIPTION

[0055] The Figures (Figs.) and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

[0056] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It is noted that wherever practicable, similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

[0057] As briefly discussed above, with the rapid development of generative audio synthesis and editing techniques, it has become relatively easy now to edit and synthesize audio content. While this democratizes the creative process and enables new applications, it also raises serious concerns regarding misuse of copyrighted data, data provenance, and authenticity. A notable example is the recent surge in deepfake audio and video. Furthermore, beyond deepfake threats, the unauthorized exploitation of copyrighted content is also a growing concern in the Al industry. Original content is increasingly exploited and modified at scale, making it difficult to track data provenance. Accordingly, the challenge of combating disinformation and verifying content authenticity in the digital age necessitates robust standards for provenance metadata. Among the various solutions to track audio provenance and guarantee artists’ protection, watermarking stands out as one of the most effective proactive approaches. In broad terms, watermarking generally involves embedding imperceptible perturbations into the audio that are both identifiable and traceable. Generally speaking, watermarking enables, among others, at least two key processes, namely: detection, which verifies the presence of the watermark in an audio file; and attribution, which involves decoding a message that uniquely identifies the original creator.

[0058] Previous audio watermarking methods have primarily focused on embedding and detecting watermarks by coupling message bit prediction with the time dimension of the audio signal. For instance, some of the existing techniques generally propose embedding watermark bits across temporal frames, requiring a prediction at each time step. These methods however increase the total number of message predictions to T times, where T denotes the length of the audio signal, leading to unnecessary computational overhead.

[0059] Furthermore, some of the existing methods are mainly designed to be robust against traditional audio preprocessing transformations such as MP3 compression, resampling, and band-based filtering. While these methods may appear to withstand such conventional distortions, they are inadequate against Al-based editing transformations. With the rise of advanced Al models capable of sophisticated audio editing, such as audio diffusion models and language models used for zero-shot unsupervised and text-based audio editing, the limitations of traditional watermarking approaches have become evident.

[0060] Additionally, in some of the existing methods, the watermarking model and the detection model are typically trained separately without any shared components, such as embedding tables. That is, the detection model would have to be independently learn to reverse the watermarking process without access to the embeddings used during watermarking. This lack of shared knowledge results in slower convergence during training and less accurate detection, especially in attribution accuracy, as the detector lacks crucial information that could facilitate the decoding process.

[0061] Some of the existing methods also propose techniques that leverage invertible neural networks (INNs) to embed watermarks into audio signals. However, while the use of INNs in these existing methods may appear to make the learning process efficient, it introduces limitations in terms of accuracy and robustness, especially against Al-based editing transformations. The invertibility constraint of INNs can restrict the expressiveness of the model, making it less capable of capturing complex transformations required for more resilient watermarking. Moreover, the reliance on INNs means that the encoder and decoder are tightly coupled through the invertible transformations, which can be both a strength and a weakness. While it ensures that the decoding process is the exact inverse of encoding, any limitations in the encoding process directly impact the decoding accuracy. This coupling can make it challenging to optimize the watermarking process for both imperceptibility and robustness against advanced transformations simultaneously.

[0062] In view of the above, there appears to be a general need to develop systems and / or methods, more specifically, neural network based watermarking techniques, that can achieve both robust detection and attribution, while at the same time allowing for efficient training of such systems. In other words, the techniques proposed in the present disclosure not only enhance learning efficiency and attribution accuracy, but also provide robustness against both traditional processing transformations and Al-based editing transformations.

[0063] References are now made to the figures. In particular, Fig. 1 schematically illustrates an example block diagram of a neural network (NN) based system 1000 for use in audio watermarking (e.g., embedding, detection, and / or decoding, etc.) according to some example embodiments of the present disclosure. It may be worth mentioning that, although the figure(s) may seem to show specific shapes / configurations of the blocks / elements / components, these are merely examples given for illustrative purposes and should not be considered to constitute a limitation of any kind. Of course, as will be understood and appreciated by the skilled person, any other suitable implementations of the system and the components thereof may be adopted.

[0064] As shown in Fig. 1, the audio watermarking system 1000 may be seen to comprise two primary components: a (watermark) generator 1100 (denoted by using Q) to embed watermark, and a (watermark) detector 1200 (denoted by using D) for recovering (detecting and / or decoding) the watermark. For the sake of simple illustration, let x E

[0065]

[0066] represent an (input) audio signal 1001 of length T and let w E {0, 1} represent a binary watermark sequence (i.e., of K bits) 1002. Depending on various implementations and / or requirements, such a watermark may be a predefined or a (e.g., randomly, or purposefully) generated (binary) sequence. For instance, in some possible examples, the watermark may be defined as a binary message with a specific meaning (e.g., potentially to be used as a hash of the content owner’s metadata). The generator 1100: IK7x {0, 1}

[0067]

[0068] may then be configured to output a watermarked signal 1003 xw

[0069]

[0070] = which should ideally preserve the audio’s perceptual quality.

[0071] On the other hand, the detector 1200 D may be generally understood to be responsible for two tasks. In general terms, first, its detection head 1230 Ddet:

[0072]

[0073] [0, 1] may be configured to produce a probability indicating whether an input contains a valid watermark, which would in turn yield a final decision of presence or absence for example with a (preconfigured or predetermined) threshold. Second, its message decoding head 1220 £>MSG: [Q, j Kmay be configurecito return a vector indicating the probability of each bit being 1 (or 0, depending on various implementations), which could in turn be thresholded to reconstruct (decode) the embedded message. It may be worth noting that one of the main reasons to have two separate heads for respective detection and decoding tasks is that the whole system can be trained end-to-end to be a lot more robust on both of the tasks. As will be understood and appreciated by the skilled person, in practical settings, the watermarked audio may undergo various transformations T (•) (e.g., in the waveform domain) such as compression, cropping, or the like, resulting in a distorted signal x„ = T (%w). Across different transformations, an ideal detector should be able to robustly detect and decode watermarks. In essence, the detection head 1230 should be able to output probabilities close to 1 for watermarked signals (and close to 0 otherwise), while the message decoding head 1220 should be able to recover a bit sequence w (sufficiently) matching the original watermark message w. Incidentally, it may be worth mentioning that, in the present disclosure, unless indicated otherwise, focus is made mainly on per-sample level detection and attribution, which may be generally considered a practical and widely adopted setting. In a broad sense, as will be understood and appreciated by the skilled person, when considering a general formulation of a neural watermarking system with detection and decoding separation, such a watermarking system may be parameterized by 0 = (0^, 0D), where Qg may be understood to denote the generator parameters, whilst 0-Dmay be understood to denote the detector parameters. During training, the detection head 1230 may typically be configured to first produce a logit for each audio input, which may then be passed through a sigmoid function to yield a probability p(x) = Dget(x) G [0, 1], For a watermarked signal x and an unwatermarked signal x, the objective of the detection head 1230 may be understood to be able to correctly classify both, often formulated by maximizing the following expected log-likelihood:

[0074] max ( E [logZ>det(x^)] + E [log(l — Z)det(xT))]Y W ®

[0075]

[0076] Bdet 'X, W, T x, T Z

[0077] where xw= (x, w) and x£ = T (xw). Then, during inference, the threshold T may usually be tuned on the validation set to control a certain level of false positive rate.

[0078] Meanwhile, the message decoding head 1220 may be configured to produce K probabilities

[0079]

[0080] Dmsg(x^) = [Dmsg(x^)1+ •■•, Dmsg(x^)K], where each entry may be understood to represent the probability that the corresponding bit is 1 (or 0, depending on various implementations, for example). Accordingly, the respective training objective may be understood to maximize the probability of correctly predicting each bit of w, which may be expressed as:

[0081] max E [wklogDmsg(x )k+ (1 - wk)log(l - Dmsg(x^)k)], (2) 0BmsgX,w, T

[0082] By thresholding each Dmsg(x^)kat 0.5, the decoded bit string w may be obtained, with the learning objective seeking w = w despite the distortion layer T applied to xw.

[0083] As further shown in the example system 1000 of Fig. 1, in some possible implementations, given audio x, the generator 1100 Q = {£gen, 7gen} may be seen to be composed of an encoder module (or simply encoder for short) 1110 (denoted by using £gen) that may be configured to encode the audio 1001 into a (compact) latent representation hx. and a decoder module (or simply decoder for short) 11207genthat may be configured to decode the latent into the watermarked perturbation 1003. In some possible implementations, the audio latent may be obtained via hx= £gen(X)

[0084]

[0085] xH, where t' = [T / a], a is a (predetermined or preconfigured) temporal downsampling factor, and H is the latent dimension. The secret message w may then be injected into the latent space with a message encoder (module) 1300 M that may be configured to map w to its latent representation according to hw= M(w Accordingly, the modulated latent waveform may be obtained by adding the message latent to the audio latent h(x, w) = hx+ hwfollowed by the decoder network 11207gento produce the predicted watermarked perturbation 1103 8W= 7gcn( (x, u / )) G BTxlwhich is later applied to the original audio 1101 to produce the watermarked audio 1104 xw= x + 8W. In some possible implementations, the decoder model 11207genmay share a symmetric structure with the encoder 1110 £gen, for example with layers of residual transposed convolution for temporal upsampling.

[0086] On the other hand, in some possible implementations, the watermark detector 1200 D = {£det> $dec} may also comprise an encoder 1210 £det. Depending on various implementations, such encoder 1210 may be configured to share a similar structure with the decoder 1110 £genused in the watermark generator 1100, but with an additional padding to match the original length T, for example. Accordingly, in some possible implementations, given a potential watermarked audio 1004 x, the encoder 1210 £detmay be configured to process the audio 1004 to obtain the respective latent representation hx= £det( w)E

[0087]

[0088] As discussed above, many of the conventional models tend to struggle to, among others, perform accurate message decoding even without distortion (See for example Table 6 of the Appendix). In broad terms, the present disclosure may be generally seen to address such issues by proposing: a cross-attention generator-detector watermarking system with a shared embedding table and temporal modulation to improve learning efficiency in message decoding; and then, to further improve watermark quality, a psychoacoustic-aligned temporal-frequency masking

[0089]

[0090] loss. These aspects will now be discussed in more detail as follows.

[0091] First, it is observed that a fully disjointed (generator-detector) architecture (e.g., Qg 0D) appears to often converge fast for the watermark detection learning, but struggles to learn the message decoding part efficiently and accurately. On the other hand, a full parameter-sharing architecture (e.g., Qg = 0D) may appear to achieve reasonable efficiency in learning both detection and message decoding, but lack enough robustness capability against various distortions. In view thereof, in a broad sense, the present disclosure generally proposes a design of a hybrid architecture with partial parameter sharing between the generator and the detector, which facilitates more effective message decoding while preserving robust capability.

[0092] In particular, in some possible implementations, it is proposed to share the message conditioning module 1300 between the generator 1100 and the detector 1200 (as illustratively shown in Fig. 1), which helps bridge the information flow between how the message is composed and how it can be reconstructed in the detector part. Specifically, the learnable part of the embedding table 1310 £ in, which may be generally seen to serve as a fundamental vector set for composing the message latent in the generator part 1100, may be utilized as a reference when decomposing the latent to retrieve the message bits in the detector 1200. To achieve this, in some possible implementations, it is proposed to leverage a cross-attention block 1221 to use the embedding table 1310 E for example as a (key-value) reference for message decoding, given the query audio latent hx.

[0093] In detail, for a given embedding table E with 2K entries and a watermarked audio xw= x + w), the goal is to reconstruct the original K embedding vectors [E^,, EIK], that were used to compose the ground-truth message latent hwand then feed them as context for the message decoding.

[0094] To achieve this, in some possible implementations, it is proposed to use an attention mechanism that may be generally understood to transform the embedding table into key and value using two different linear projections, and perform attention-based vector merging with a query from the reconstructed latent representation hx.

[0095] Specifically, since the two nearby entries in E may be generally understood to represent one position with different bit states, merging them into one 2H-dim vector may represent the latent of each position. This reshaped embedding matrix E' E ^Kx2Hmay be transformed into K, V with two linear projections WK, WvE IK2HxHthat is K = E'WKand V = E'WV.

[0096] Then, the reconstructed waveform latent may be demodulated along the temporal axis hxE, to obtain the first version of raw prediction Vxfor the original K components. To do this, in some possible implementations, it may be first used a linear projection WdemE BTx / <for obtaining the query sequence hxem= WdemhxE mKxHand followed with a linear query projection WQE BHxH. Afterwards, the embedding table E may be utilized to further refine the final component prediction Vxwith the following cross-attention mechanism:

[0097] Q = hxemWQE ^KxH, (3) K = E'WKE ^KXH, V = E'WVE ^KxH, nvT\ _ | E RKxK,

[0098]

[0099] VH J Vx= act(AV) E ^KXH, where H is the latent dimension, K is the number of message bits, T is the temporal dimension, and act(-) may be any suitable activation function. In some possible implementations, exponential linear units (ELU) based (or any other suitable) techniques may be used.

[0100] With the predicted Vx, a linear projection layer WdecE BHxlwith sigmoid activation function may be further employed for the construction of the message decoding head 1220. The final logit for the message prediction bit may then be obtained as wx=

[0101]

[0102] o^KWrfec), where cr(-) may be a suitable sigmoid activation function.

[0103] In some of the conventional techniques, the message latent hwmay be obtained by a so-called mean-pooling operation with temporal-axis repetition according to for example hw= Repeat., t'^ E. For instance, in some possible examples, an embedding

[0104]

[0105] table with 2K entries, E E B2 / <xH, may be used to encode a total of 2Kpossible message states. Specifically, each bit Wj E {0,1} may be mapped to index Ij = 2j + Wj, allowing separate representations for 0 / 1 states in each bit position with distinct embedding vectors. Given the position sequence p = [ / -..., / K], the retrieved K embedding vector sequence may be denoted as U(w) = [

[0106]

[0107] E^,..., EIK] E B / < XH. Then, an average pooling operation on f(w) may be applied to obtain the final message latent hw

[0108]

[0109] = -Sf=i E;. G The message latent may then be repeated along the temporal axis and then added to the waveform latent, leading h(x, w) = hx+ hw. In simple words, such approach may be viewed as injecting the message information mostly into the frequency domain, which may however limit the message hiding capabilities.

[0110] To improve upon this, in some possible implementations, particularly when knowing the input audio length T, it may be introduced a temporal message conditioning mechanism that could employ a linear modulation layer WME B / <xtto obtain the message latent according to the following:

[0111] h

[0112]

[0113] w= W^V(w) E ^'XH. (4) As will be understood and appreciated by the skilled person, this simple design not only prevents the sole reliance on the frequency domain for distributing the / Gbit message, but also significantly facilitates the learning process for message decoding.

[0114] Furthermore, achieving imperceptibility may also be understood to represent one of the key requirements of a watermarking system. For instance, in some possible cases, it may be understood that a watermark might exploit the perception characteristics of the human auditory system so that the artifacts remain imperceptible. In this regard, it is noticed that while some of the prior work may have tried to introduce perceptual losses such as TF-Loudness based techniques or the like, these often adopt a coarse approach based on loudness differences within fixed temporal-frequency (or time-frequency, TF) tiles. As a result, such methods may be understood to typically face two main limitations: (i) they may not fully capture sophisticated auditory masking effects, particularly the interactions between masker and maskee signals across different tiles, and (ii) loudness differences alone may provide weak supervisory signals for guiding the watermark embedding, resulting in a coarse penalty distribution.

[0115] In view thereof, the present disclosure generally proposes a more psychoacoustic-aligned TF-masking loss, which may be understood to employ a per-tile penalty with asymmetric temporal-frequency decay.

[0116] In detail, let S0(m, t) denote the magnitude of the original audio’s mel-spectrogram, and the watermarked Mel-spectrogram may be denoted as Sw(m, t). In some possible implementations, in order to identify those strong “masker” tiles, a magnitude threshold asmay be applied to different timestamps on each frequency band (as= 0.8) according to for example:

[0117] Mnasker = {O, t) I So(m, t) > asmaxSo(m, t')}. (5)

[0118]

[0119] In some possible implementations, each index pair (mc, tc) Emaskermay be seen to act as a masker, and its masking influence over neighboring tiles may be modeled with a linear energy decay in the decibel domain, for example. Specifically, in some possible implementations, given mel-scale frequency radius r™ and r™, and time-axis radius and r^, the local region of a masker may be defined as:

[0120] R

[0121]

[0122] (mc, tc) = {(m, t) | — r™ < Am < r™, —r£ < At < (6)

[0123] where Am = m — mcand At = t — tc. In some possible implementations, aligned with the empirical studies that the post-masking region (forward masking) may be understood to be usually longer than the pre-masking region (backward masking) in the temporal axis, asymmetric radii

[0124]

[0125] and may be set, with a forward masking region of for example 200 ms and a backward masking region of 20 ms, for instance. On the other hand, for the frequencybin radius, an empirical critical bandwidth for each mel-bin may be first computed using a formula according to for example W (mc= 25 + 75 ( 1 + 1.4 ), where

[0126]

[0127] T (mc) may denote the approximated center frequency obtained by converting each Mel bin V[mc]

[0128] value F[mc] back to Hz using for example T (mc) = 700 • (lOzsoos — 1), or the like. Then the radii for each mel-bin may be set as r™ = r™ = r™ • y, where y = W (mc) / | T (m — ^(m0)| may be understood to represent a suitable adaptive scaling term, V[mi] may denote the Z-th mel-bin value, r™ may denote a base radius that may for example be set to be 3 (or any other suitable value). With the dynamic radius scaled by the critical bandwidth - where a higher frequency component may be generally understood to have a wider masking range - a better performance than using constant y is observed.

[0129] Generally speaking, the physical structure of the cochlea determines that lower frequencies (located at the base of the cochlea) may cause broader and stronger excitation patterns compared to the higher frequencies (located at the apex). In some possible implementations, to inject this bias of slower decay in the upward spread of masking, the decaying term for the upward direction may be set to be smaller than the one in the downward direction, i.e., a < a. On the other hand, for the temporal decay, since the forward decay is typically slower, the forward decaying slope may be set less steep than the backward one, i.e., t < p.

[0130] Accordingly, in some possible implementations, given a masker (mc, t, the threshold energy for the masker tile (m, t) may be computed as:

[0131] ^rnaskC^h Z, (mc, ^c)) 201og 10<^O (mc,tc) - |AE|, (7) w

[0132]

[0133] here AE = o^"max(0, Am) + ay min(0, Am) + / ?t+max(0, At) + / ?t-min(0, At)

[0134] In some possible implementations of psychoacoustic modeling, particularly in audio compression, the global masking threshold may be determined by considering only the most dominant masker at each frequency component. Following this, the final masking threshold ^mask(m> may be computed as:

[0135] £mask(m, t) = max Fmask(rn, t; (mc, tc)). (8)

[0136] (mc,tcyMmask,

[0137] Generally speaking, this ensures that each maskee tile receives masking energy from its most perceptually dominant masker, while weaker maskers have negligible impact. As illustrated above, psychoacoustic masking may be understood to apply effectively only when the masker’s masking energy surpasses the tile’s energy. Accordingly, in some possible implementations, the masking threshold may be applied on the watermarked mel-spectrogram at 201og10<S’W / (m, t), filtering the masked tiles set as:

[0138] •M'mask = {( n, t) I EmaskO- > 201og10^(m, t)}. (9) In some possible implementations, in order to encourage the model to embed the watermark in those maskee regions, per-tile penalty terms may be designed which weight the2difference between the watermarked and original audio in Mel-spectrogram space according to:

[0139] X(m, t) = 1 + l(m,t)eMmask• 10Fmask(^4) / 2Oj(IQ)

[0140] _ V OII2 (n)TF2-i A(m, t)

[0141]

[0142] (m, t) where 10Fmask04) / 2o denotesthe masking energy, and the weighting term A m, t) may be generally set to be larger for tiles in the masking regionmask, effectively allowing more watermark signal to be embedded in those locations. On the other hand, for non-masked tiles, A(m, t) = 1 may generally enforce standard i2loss.

[0143] As will be understood and appreciated by the skilled person, the above-illustrated psychoacoustic-aligned temporal-frequency masking loss is merely one of the possible implementations. Of course, any other suitable loss function (s), perceptual based or not, may be adopted (in isolation or combination) as well, depending on various implementations and / or circumstances.

[0144] For instance, any one (or more) of the following may be considered as suitable auxiliary (perceptual) loss for purposes of the training. In some possible examples, a - / ’1constraint may be used on the watermark signal 8Xto ensure waveform-domain smoothness. In some possible examples, a multi-scale Mel spectrogram loss Tmsspecmay be incorporated to manage frequency-domain fidelity. Furthermore, an adversarial loss £advmay be adopted on multi-scale Short Time Fourier Transform (STFT) spectrograms for perceptual improvement. Notably, in some possible implementations, a lower weightadv= 1 may be used. One of the main reasons behind this may be understood that aggressively pushing the watermarked audio distribution to be close to the clean audio distribution in an adversarial sense may penalize desirable watermark characteristics (e.g., certain musical “remixes” can produce satisfactory imperceptible watermarks, but still yield high adversarial loss). Accordingly, it is observed that lowering the adversarial weight would generally allow the model to focus more on the psychoacoustic aspects to develop a human-centric, imperceptible watermark.

[0145] It may be worth noting that the proposed system may be further extended, for example to perform watermark localization. In some possible implementations, this may be achieved by employing a sliding window detection mechanism. For instance, assuming that the proposed model operates on 1-second audio segments and incorporates shifting-robust transformations, the per-segment detection probability may be distributed to a finer, per-frame level by utilizing multiple overlapping detection windows. For instance, in some possible implementations, let the audio signal be represented as a sequence of frames F =

[0146]

[0147] The proposed localization approach may utilize multiple overlapping detection windows, where each window I / Fkcorresponds to a Is audio segment. For each window

[0148]

[0149] the proposed model may compute a watermark detection probability P(M|VI / k), where M denotes the presence of the watermark. A given frame ftmay be part of several such overlapping windows, let this set be denoted as W( j) = {I / Fk| ftE Wk}. The per-frame detection probability

[0150]

[0151] may then be determined by aggregating the probabilities from all windows covering that frame. Then, the average of P(M|VI / k) for all E may be taken as the per-frame detection probability P(M\ft, i.e.,

[0152] 1 (12) P(M | fd = | Wk). IWDI

[0153]

[0154] It is observed that this process may effectively distribute the segment-level detection scores to a finer, per-frame resolution, enabling precise localization of the watermarked regions. Next, Fig. 2 is a flowchart illustrating an example of a method 2000 illustrating an example of a method of training a neural network based system for use in audio watermarking according to some example embodiments of the present disclosure. The neural network based system may for example be analogous or similar to the system 1000 as shown in Fig. 1. For instance, as illustrated above, the neural network based system may comprise: a neural network based watermark generator (e.g., the watermark generator 1100 of Fig. 1) for embedding a watermark (e.g., the watermark message 1002 of Fig. 1) into an input audio signal (e.g., the input audio 1001 of Fig. 1); and a neural network based watermark detector (e.g., the watermark detector 1200 of Fig. 1) that comprises a detection head (e.g., the watermark detection head 1230 of Fig. 1) for determining presence of a watermark in a target audio signal and a decoding head (e.g., the message decoding head 1220 of Fig. 1) for decoding the watermark from the target audio signal.

[0155] More particularly, the method 2000 may comprise, at step S2100, obtaining, as a training input (or in other words, input(s) for the training), a test audio signal and a test watermark sequence that are used as ground truth.

[0156] The method 2000 may further comprise, at step S2200, embedding, by using the watermark generator and based on a learnable embedding table (e.g., the embedding table 1310 of Fig.

[0157] 1), the test watermark sequence into the test audio signal, thereby obtaining a watermarked audio signal.

[0158] The method 2000 may yet further comprise, at step S2300, predicting, based on the watermarked audio signal, the presence of a watermark by using the detection head, and an estimated watermark sequence by using the decoding head.

[0159] In addition, the method 2000 may comprise, at step S2400, iteratively training the neural network based system based on at least one loss function indicative of the respective difference between the predicted presence, the estimated watermark sequence, and the ground truth. As already discussed in detail above, in some possible implementations, data augmentations of the watermarked audio may be performed before running the detector during training. As will be understood and appreciated by the skilled person, by augmenting the watermark with different modifications during training, the watermark may be made robust to the same type of modifications when it is used at inference time.

[0160] More particularly, the method 2000 may further comprise, at step S2500, sharing the embedding table between the watermark generator and the watermark detector, such that the prediction of the estimated watermark sequence may be based on the embedding table that has been used for the embedding.

[0161] Configured as proposed, the present disclosure generally provides an efficient, flexible yet reliable mechanism of training a (neural network based) system for use in audio watermarking processes (e.g., embedding, detection, decoding, etc.) which achieves improved performance in terms of efficiency, robustness against wide range of audio transformations, etc., in both (watermark) detection and attribution, while at the same time also preserving perceptual quality. Once training is finished, the trained system may then be used or operated for inference, such as watermark embedding, watermark detection, and / or watermark decoding, etc. For example, Fig. 3 is a flowchart illustrating an example of a method 3000 of a neural network based system for audio watermarking according to some example embodiments of the present disclosure. The neural network based system may for example be analogous or similar to the system 1000 as shown in Fig. 1. That is, the neural network based system may comprise: a (neural network based) watermark generator for embedding a watermark into an input audio signal based on a learnable embedding table; and a (neural network based) watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal. In particular, the embedding table is shared between the watermark generator and the watermark detector. Also, the system may have undergone the training process 2000 as illustrated for example with respect to Fig. 2. Thus, repeated description thereof may be omitted for reasons of conciseness.

[0162] In particular, the method 3000 may comprise, at step S3100, obtaining the input audio signal. The method 3000 may then comprise, at steps S3200, obtaining the watermark. Finally, the method 3000 may comprise, at step S3300, embedding, by using the watermark generator and based on the embedding table, the watermark into the input audio signal, thereby obtaining a watermarked audio signal.

[0163] Next, Fig. 4 is a flowchart illustrating another example of a method 4000 of a watermark detector for audio watermark detection and decoding according to some example embodiments of the present disclosure. The watermark detector may for example be implemented in analogy or similar to the watermark detector 1200 as shown in Fig. 1, or any other suitable manner. Accordingly, the watermark detector may comprise a detection head and a decoding head. Also, the watermark detector may have been properly trained (e.g., as a part of the training process 2000 of Fig. 2). Thus, repeated description thereof may be omitted for reasons of conciseness.

[0164] In particular, the method 4000 may comprise, at step S4100, obtaining a target audio signal. The method 4000 may also comprise, at step S4200, determining, by using the detection head, the presence or absence of a watermark in the target audio signal. Moreover, the method 4000 may comprise, at step S4300, decoding, by using the decoding head, the watermark from the target audio signal based on a learnable embedding table. More particularly, the embedding table may have been used by a watermark generator during training for embedding a watermark into an input audio signal.

[0165] In summary, the present disclosure generally introduces a blended / hybrid architecture of partial parameter sharing between generator and detector, which jointly achieves both efficient learning and robustness. More specifically, the present disclosure proposes a crossattention module that leverages a shared embedding table to facilitate message decoding in the detector part. Further, the present disclosure proposes a simple yet effective conditioning mechanism that distributes the message temporally before injection, which further improves the learning efficiency. With these key components, a significant boost in both robust detection and attribution performance is observed. Moreover, to enforce stronger constraints for watermark quality (imperceptibility), the present disclosure also introduces a per-tile temporal-frequency masking loss. In simple terms, masking energy with an asymmetric 2D kernel, identifying the masked regions is first computed, and then the masking energy is used as a weighting factor for computing a TF-weighted loss in the mel- spectrogram domain. With these efforts, robustness across a wide range of audio editing transformations is achieved while preserving superior perceptual quality. Furthermore, under the more challenging task of generative model editing, the techniques proposed in the present disclosure also demonstrate watermark detection capability even when edits of strong strength are applied (e.g., testing in a zero-shot manner on unseen generative editing transformations).

[0166] For the sake of completeness, it is noted that the present disclosure also covers the following aspects. As will be understood and appreciated by the skilled person, the following aspects may be used, either in isolation or in combination with any one or more of the aspects (or possible implementations) already illustrated above. That is, the aspects discussed below should not be seen to constitute a limitation of any kind, but should instead to understood in a manner that suitably supplements or complements those already discussed above.

[0167] In particular, the techniques, such as systems and methods, for audio watermarking described herein may be generally understood to introduce a comprehensive approach to improve audio watermarking by addressing the limitations / issues in existing watermarking approaches. The following are some of the key aspects.

[0168] First, in accordance with some example embodiments, message prediction may be decoupled from the time dimension. This generally addresses the problem that, in some conventional techniques, the coupling of message bit prediction with the time dimension may increase computational complexity and hamper learning efficiency.

[0169] More particularly, to address this, the proposed techniques include splitting and decoupling message prediction from the time dimension. By treating the prediction of watermark bits independently of the audio’s temporal length, the total number of required message predictions is significantly reduced. Moreover, this decoupling allows for more efficient learning and faster computation, as the model no longer needs to process redundant information tied to each time frame.

[0170] Second, in accordance with some example embodiments, there may also be proposed shared learning between watermarking (watermark embedding) and detection models via embedding table and cross-attention. This generally addresses the problem that, in some conventional techniques, the coupling of message bit prediction with the time dimension may increase computational complexity and hamper learning efficiency.

[0171] More particularly, for watermarking model enhancements, it is proposed to learn the embedding table during the internal processing of embedding watermark bits into the audio signal. On the other hand, for detection Model Enhancements, it is proposed to receive the embedding tables from the watermarking model. Then, the cross-attention module is implemented to reference these embeddings during detection. Configured as such, particularly by sharing the embedding tables and utilizing cross-attention mechanisms, the detection model gains direct access to the embeddings used in watermarking. This shared learning accelerates the detector’s ability to reverse-engineer the hidden messages, enhancing learning efficiency and attribution accuracy.

[0172] Third, in accordance with some example embodiments, the present disclosure may also achieve robustness against Al-based editing transformations via Principal Component Analysis (PCA) in noise space, which will be illustrated in more detail below. This generally addresses the problem that traditional / existing watermarking methods are typically not robust against Al-based editing transformations that can significantly alter the audio signal using advanced Al models such as audio diffusion models and language models. Moreover, a naive approach to counter this, which involves sampling over all possible editing transformations, is computationally infeasible due to the vast space of possible transformations.

[0173] More particularly, the proposed techniques may be understood to include a method that enhances robustness against Al-based editing transformations by focusing on the major editing directions in the noise space. In some possible implementations, it employs sub-space iteration to compute the principal components representing the primary directions of variation introduced by Al-based audio editing models. By concentrating on these principal components during training, the watermarking model learns to embed watermarks that are resistant to alterations along these major editing directions.

[0174] In more detail, during training, the model may perform sub-space iterations to find the principal components in the noise space associated with Al-based editing transformations. These principal components may be understood to capture the most significant variations that Al editing introduces. The model may then incorporate these principal components into the training process, adjusting the watermark embedding to be robust against changes along these directions. Configured as such, particularly by focusing on the primary editing directions, the model may achieve robustness against Al-based editing transformations more efficiently than methods that attempt to sample over all possible transformations. In simple words, this approach enhances the watermark's resilience while maintaining computational efficiency. Fourth, in accordance with some example embodiments, the present disclosure may further propose a symmetric architecture with learnable message composition.

[0175] In particular, concerning the symmetric architectural design, both the watermarking and detection models may be designed with matching hidden dimensions in their crucial bottleneck layers. Such symmetric architecture may be understood to allow the detection model to effectively learn the inverse function of the watermarking model. Further, matching hidden dimensions may facilitate better reconstruction of the original messages and more accurate watermark detection. Moreover, concerning learnable mapping function for message composition, instead of directly averaging the hidden states of the message embeddings, the proposed techniques may introduce a learnable mapping function. This mapping function may be trainable and adapt during the learning process to optimally compose message embeddings. As a result, the adaptive nature of the learnable mapping may enhance the performance and speed of the reverse decoding process in the detection model, leading to improved attribution accuracy and robustness against variations in the audio signal.

[0176] Next, as a brief summary, the proposed system 1000 as illustrated above with respect to Fig.

[0177] 1 may be understood to generally include the following components:

[0178] 1. Watermark Generator (or simply referred to as watermarking) Model: Input: The original audio signal and the message bits to be embedded.

[0179] Process:

[0180] Embedding Table: Learns embedding tables for the message bits, mapping discrete bits to continuous vector representations. In other words, message bits may be mapped to high-dimensional vector representations; and the embedding tables may be learned during training to optimize embedding efficiency.

[0181] Learnable Mapping Function: Uses a trainable function to compose the hidden states of the message embeddings, allowing the model to adaptively learn optimal compositions. For instance, in some possible examples, this may be achieved by a neural network layer (e.g., fully connected layer) that composes the message embeddings and may be adjusted during training to find the optimal way to combine embeddings.

[0182] Decoupled Prediction: Separates message prediction from the time dimension, reducing computational complexity.

[0183] Robustness Enhancement against traditional preprocessing: Performs various processing steps such as MP3 compression and band-based filtering to enhance the watermarking robustness.

[0184] Robustness Enhancement via Principal Component Analysis: Performs sub-space iterations during training to compute principal components in the noise space representing major editing directions of Al-based audio editing models. Incorporates / Integrates these into the training process to enhance robustness (e.g., to adjust the watermark embedding for robustness against changes along these major editing directions).

[0185] Symmetric Hidden Layers: Employs (multiple) hidden layers with specific dimensions that match those in the detection model (symmetric architecture).

[0186] Output: Watermarked audio signal with embedded messages that can be retrieved with the detector. In particular, in some possible implementations, an output layer may be configured to generate the watermarked audio signal by combining the original audio with the embedded message. 2. Watermark Detector (or simply referred to as Detection) Model:

[0187] • Input: Received audio signal (which may or may not contain a watermark).

[0188] • Process:

[0189] Shared Embedding Table: Utilizes the embedding tables learned by the watermarking model. In particular, the embedding tables may be received from the watermarking model, ensuring consistency in message representation.

[0190] Cross-Attention Module: Implements cross-attention mechanisms to reference the embeddings during detection, focusing on relevant parts of the input. In particular, the cross-attention module may be configured to focus on relevant parts of the input signal by referencing the embeddings, thereby enhancing the model’s ability to detect and reconstruct embedded messages.

[0191] Symmetric Hidden Layers: Mirrors the hidden dimensions of the watermarking model (symmetry) for efficient inverse processing. This generally facilitates efficient inversion of the watermarking process.

[0192] Learnable Decoding: Employs a trainable function to reverse the message composition process.

[0193] Decoupled Detection: Independently processes message detection without being tied to the time dimension. This decoupling significantly boosts efficiency.

[0194] • Output: Decoded message bits and a binary prediction of whether the input audio is watermarked. In particular, in some possible implementations, an output layer may be configured to output the detected message bits and indicate the presence of a watermark. Specifically, these two predictions may be output in two separate heads. In the following, possible designs in the efficient Al-editing augmentation sampling via PCAs will be schematically illustrated in detail.

[0195] a. Understanding the Noise Space and AI-Based Editing, in accordance with some possible embodiments:

[0196] Al-based audio editing models, such as those using Denoising Diffusion Probabilistic Models (DDPMs), perform edits by manipulating the noise space of the audio signal. During the generative process, these models iteratively denoise a noisy signal to produce the edited audio.

[0197] b. Extracting Principal Components in Noise Space, in accordance with some embodiments: To make the watermark robust against Al-based editing transformations, the watermarking model needs to understand the primary directions in the noise space along which these edits occur.

[0198] • Principal Component Analysis (PCA): PCA is used to identify the directions (principal components) in the noise space that capture the most significant variations introduced by Al-based edits.

[0199] • Subspace Iteration Method: Due to the high dimensionality of the noise space, computing principal components directly is computationally expensive. Subspace iteration is an efficient numerical method to approximate the principal components. c. Steps for Computing Principal Components, in accordance with some embodiments:

[0200] 1. DDPM Inversion:

[0201] Given an original audio signal x0, DDPM inversion is used to map it to a sequence of noise vectors {xT, zT,..., z^.

[0202] The inversion process retrieves the noise vectors that would generate x0when passed through the DDPM.

[0203] 2. Computing Posterior Covariance Matrix:

[0204] At each time step t in the diffusion process, the posterior distribution P(x0|xt) is considered, where xtis the noisy signal at step t.

[0205] The posterior covariance matrix Cov[x0| t] captures the uncertainty of x0given xt.

[0206] 3. Subspace Iteration for Principal Components:

[0207] Initialization: Start with a set of random vectors as initial guesses for the principal components.

[0208] Iteration: • Perturbation: At each iteration k, perturb xtalong the current estimate of the principal component

[0209]

[0210] to obtain xshifted.

[0211] • Denoising: Use the denoising function / xt(xshifted) to predict the original signal x0|t from xshifted.

[0212] • Update: Compute the difference between x0|tand the unperturbed prediction to update the estimate of the principal component.

[0213] • Orthogonalization and Normalization: Ensure the updated vectors remain orthogonal and normalized.

[0214] Convergence: Repeat until the principal components converge.

[0215] 4. Integrating Principal Components into Training:

[0216] Once the principal components {v^t] are computed, they represent the major directions along which Al-based editing transformations modify the audio signal.

[0217] During watermark embedding, the model adjusts the embedding process to be robust against perturbations along these directions.

[0218] Specifically, the watermarking model can incorporate perturbations along the principal components into the training data, simulating the effects of Al-based edits.

[0219] d. Enhancing Robustness in Watermark Embedding, in accordance with some embodiments:

[0220] • Adversarial Training with Principal Components:

[0221] The model trains to minimize the difference in watermark detection accuracy between the original and edited audio.

[0222] By incorporating the principal components into the adversarial training process, the model learns to embed watermarks that are invariant to changes along these major editing directions.

[0223] • Mathematical Formulation:

[0224] The edited audio sample at time step t can be expressed as: ■

[0225]

[0226] ^t-1 3” ^i)tf 3” &tzt

[0227] Where:

[0228] •

[0229]

[0230] is the denoising function at time t.

[0231] • y is a user-defined parameter controlling the strength of the adjustment.

[0232] • ctadjusts the perturbation based on the time step.

[0233] 1 / 2

[0234] •

[0235]

[0236] is the square root of the eigenvalue corresponding to the principal component v^,.

[0237] • Vt\t, is the principal component computed at time t'.

[0238] • <jtztrepresents the added noise at time t.

[0239] Result: By focusing on the primary editing directions, the model achieves robustness against Al-based editing transformations more efficiently than methods that attempt to sample all possible transformations. This approach enhances the watermark's resilience while maintaining computational efficiency.

[0240] The techniques for watermarking described herein may be understood to have broad applicability in various areas requiring secure and efficient audio watermarking, including (but not limited to) for example:

[0241] 1. Content Authentication and Copyright Protection:

[0242] a. Embedding unique identifiers into audio content to verify ownership and prevent unauthorized distribution.

[0243] 2. Attribution in Media and Communications:

[0244] a. Tracing the origin of audio files in broadcasting, streaming services, and social media platforms.

[0245] 3. Secure Communications:

[0246] a. Embedding confidential messages within audio signals for covert communication.

[0247] 4. Digital Rights Management (DRM): a. Enforcing licensing agreements by embedding usage rights information into audio content.

[0248] 5. Audio Forensics:

[0249] a. Assisting in the verification of audio authenticity in legal and investigative contexts.

[0250] Of course, any other suitable use case / application may be exploited as well, as will be understood and appreciated by the skilled person.

[0251] For the sake of completeness, in the following, some of the possible experiments and comparison results will be exemplarily described with reference to some of the figures.

[0252] Further discussions may also be found in the Appendix section. Needless to say, the description as follows and further in the Appendix is merely provided for illustrative purposes, and should in no way be seen to constitute a limitation of any kind, as will be understood and appreciated by the skilled person. For instance, it should be understood that any other suitable training data sets and / or evaluation means may be adopted, depending on various implementations and / or requirements.

[0253] Experimental Setup

[0254] Following some of the prior works, a sampling rate of 16kHz and one-second mono samples may be used for training (T = 16000) under 16 diverse audio editing transformations. The models may be trained on a mixed audio dataset of 4100 hours containing speech (3016 hours VoxPopuli and 100 hours EibriSpeech), music (9 hours MusicCaps and 880 hours Free Music Archive), and sound effects (98 hours AudioSet). It goes without saying that any other suitable sampling rate (e.g., up to 44.1 kHz or even higher) may be applicable as well, depending on various implementations and / or circumstances. Similarly, any other suitable audio singles apart from music (e.g., speech or the like) may be applicable as well, as will be understood and appreciated by the skilled person. For evaluation, a held-out test set from MusicCaps of size 100 may be used. For each audio file, 100 distinct messages may be embedded, resulting in 10k unique watermarked audio samples. Each of these samples may then be subjected to 16 different audio transformations (e.g., in the waveform domain), leading to a total of 160k evaluated instances. The audio duration may be set as 5s by default in evaluation. The loss weights may be set as: ATF= 1,adv=

[0255]

[0256] 1, = 0.1,msspec= 2, ^detect = ^message = 10- The Adam optimizer with learning rate le -

[0257]

[0258] 5, = 0.4,2= 0.9, and Exponential Moving Average (EMA) with a decay factor of 0.99 updated every step may be used. The training may use a batch size of 16 for a total of 73k steps, and the model latent dimension may be H = 32. To boost the sampling of the transformation efficiency, the sampling probability of each transformation may be updated every 1000 steps on the validation set, adjusting it based on the validation accuracy of each transformation.

[0259] For the attribution experiment, the simulation protocol that defines a message pool of size N (G {100,1000,10000}) may be followed, where each message may be uniquely associated with a different user. As N increases, the message length (in bits) may also grow, making attribution more challenging due to the increased complexity of distinguishing individual messages. During decoding, the message may be retrieved from the pool by selecting the one with the closest Hamming distance. Attribution accuracy may be defined as the fraction of correctly attributed messages among all audio inputs that were positively detected as watermarked. Four state-of-the-art baselines: AudiowMark, WavMark, Timbre WM, and AudioSeal may be compared to. Please refer to Appendix A for more details.

[0260] Detection and Attribution Effectiveness

[0261] Robustness to Standard Edits. The detection and attribution performance across a comprehensive suite of standard audio editing operations is presented in Table 1. For detection, both the overall accuracy and the true / false positive rates (TPR / FPR) are reported. For attribution, the averaged performance across different user numbers is reported (see Fig.

[0262] 7 for the decomposed result). The proposed method achieves new state-of-the-art performance on both tasks, maintaining high detection accuracy (99.19% average) and attribution accuracy (93%) on average across transformations. Notably, while AudioSeal achieves strong detection performance (97.1% average), it struggles to perform effective attribution (39% average). The traditional approach, AudiowMark, exhibits more balanced detection-attribution trade-offs (around 88% attribution) but lower overall detection performance (92.9% average). Under the speeding transformation, the proposed method achieves 99.5% detection accuracy while all other methods degrade to near random-guess levels (50-61%). While attribution remains challenging under speed change, it is shown that the attribution performance can be partially restored via a speed-reversion layer (see Appendix C.3.1 for details). Fig. 7 further validates that the proposed technique consistently maintains high attribution accuracy even with increasing message pool sizes. Furthermore, statistical tests are conducted to validate the improvement significance in attribution across different transformations, please refer to Appendix C.14 for more details. In summary, it is demonstrated that the proposed technique is more robust under standard edits for both detection and attribution. See Appendix C.3 for more details.

[0263] AudioMark WavMark TimbreWM AudioSeal Proposed Technique Det. Att. Det. Att. Det. Att. Det. Att. Det. Att. Edit (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) Identity 1.00 1.00 1.00 1.00 0.995 0.93 1.00 0.69 0.995 1.00 (1.00 / 0.00) (1.00 / 0.00) (0.99 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Bandpass 1.00 1.00 1.00 1.00 0.985 0.93 1.00 0.31 0.995 0.99 (1.00 / 0.00) (1.00 / 0.00) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Boost 1.00 1.00 1.00 1.00 0.98 0.91 1.00 0.50 0.995 1.00 (1.00 / 0.00) (1.00 / 0.00) (0.96 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Duck 1.00 1.00 0.995 1.00 0.97 0.89 1.00 0.56 0.995 1.00 (1.00 / 0.00) (0.99 / 0.00) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Echo 1.00 1.00 1.00 1.00 0.945 0.91 1.00 0.38 0.995 0.99 (1.00 / 0.00) (1.00 / 0.00) (0.89 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Highpass 1.00 1.00 0.95 1.00 0.985 0.93 1.00 0.31 0.995 1.00 (1.00 / 0.00) (0.90 / 0.00) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Lowpass 1.00 1.00 1.00 1.00 0.985 0.90 1.00 0.56 0.995 1.00 (1.00 / 0.00) (1.00 / 0.00) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) MP3 0.94 1.00 0.805 1.00 0.95 0.86 1.00 0.38 0.995 1.00 (0.89 / 0.00) (0.61 / 0.00) (0.91 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Pink 1.00 1.00 1.00 1.00 0.995 0.92 1.00 0.75 0.995 1.00 Noise (1.00 / 0.00) (1.00 / 0.00) (0.99 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Random 1.00 1.00 1.00 1.00 0.975 0.90 1.00 0.56 0.995 1.00 Noise (1.00 / 0.00) (1.00 / 0.00) (0.95 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Smooth 1.00 1.00 0.96 0.88 0.98 0.80 1.00 0.19 0.995 1.00 (1.00 / 0.00) (0.92 / 0.00) (0.96 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Speed 0.50 0.00 0.50 0.00 0.515 0.18 0.61 0.00 0.995 0.03 (0.60 / 0.00) (0.00 / 0.00) (0.07 / 0.04) (0.36 / 0.15) (0.99 / 0.00) Resample 1.00 1.00 1.00 1.00 0.98 0.92 1.00 0.56 0.995 1.00 (1.00 / 0.00) (1.00 / 0.00) (0.96 / 0.00) (0.99 / 0.00) (0.99 / 0.00) AAC 1.00 1.00 1.00 1.00 0.975 0.90 1.00 0.12 0.995 0.88 (1.00 / 0.00) (1.00 / 0.00) (0.95 / 0.00) (0.99 / 0.00) (0.99 / 0.00) EnCodec 0.50 0.00 0.805 0.00 0.625 0.07 1.00 0.31 0.965 0.99 (nq=16) (0.60 / 0.00) (0.61 / 0.00) (0.50 / 0.25) (0.99 / 0.00) (0.93 / 0.00) Crop 0.965 1.00 0.995 1.00 0.96 0.85 1.00 0.12 0.975 1.00 (0.93 / 0.00) (0.99 / 0.00) (0.92 / 0.00) (0.99 / 0.00) (0.98 / 0.03) Average 0.929 0.88 0.918 0.87 0.925 0.80 0.971 0.39 0.9919 0.93 (0.859 / 0.000) (0.836 / 0.000) (0.869 / 0.019) (0.950 / 0.010) (0.9856 / 0.0019)

[0264]

[0265] Table 1. The accuracy of detection and attribution across different editing operations. For detection, it is also reported the True Positive Rate (TPR) and False Positive Rate (FPR) where the threshold is selected by Youden’s Index on a balanced validation set.

[0266] Robustness to Generative Edits. Beyond the standard audio transformations that are seen during training, audio generative editing is one particularly challenging transformation that watermarking systems might have to endure at deployment. To simulate this, two state-of-the-art audio generative models, AudioLDM2 and Stable Audio are used, with a text-guided DDIM inversion method proposed in ZETA. Various editing strength { E {10,70,110} is tested, which represents the diffusion forward step when using DDIM inversion. As shown in Table 2, AudiowMark and WavMark degrade to a random-guess level performance, while Timbre WM and AudioSeal show inferior performance across different editing strengths, around 50-60%. In contrast, the proposed method maintains consistently high detection accuracy, averaging 91-94% across all editing strengths, and is consistent across both generative models. While the proposed technique demonstrates strong detection robustness (e.g., >90% accuracy) against generative editing attacks in a zero-shot manner, achieving similarly robust attribution performance under such strong edits remains a challenge and an important direction for future work. Nevertheless, the detection performance of the proposed technique marks a significant advancement. This demonstrates that the proposed technique generalizes better to unseen generative edits compared to existing methods. Please refer to Appendix C.4 for more details. Proposed System t AudiowMark WavMark Timbre WM AudioSeal

[0267] Technique 0.50 0.50 0.47 0.69

[0268] 10 0.94 (0.94 / 0.06) (0.50 / 0.50) (0.50 / 0.50) (0.81 / 0.88) (0.63 / 0.25)

[0269] Stable 0.50 0.50 0.53 0.59

[0270] 70 0.91 (0.88 / 0.06) Audio (0.50 / 0.50) (0.50 / 0.50) (0.19 / 0.13) (0.75 / 0.56)

[0271] 0.50 0.50 0.53 0.59

[0272] 110 0.91 (0.88 / 0.06) (0.50 / 0.50) (0.50 / 0.50) (0.56 / 0.50) (0.94 / 0.75)

[0273] 0.50 0.50 0.53 0.59

[0274] 10 0.94 (0.94 / 0.06) (0.50 / 0.50) (0.50 / 0.50) (0.81 / 0.75) (0.94 / 0.75)

[0275] Audio 0.50 0.50 0.50 0.59

[0276] 70 0.94 (0.94 / 0.06) LDM2 (0.50 / 0.50) (0.50 / 0.50) (0.13 / 0.13) (0.94 / 0.75)

[0277] 0.50 0.50 0.50 0.59

[0278] 110 0.94 (0.94 / 0.06) (0.50 / 0.50) (0.50 / 0.50) (0.00 / 0.00) (0.94 / 0.75)

[0279]

[0280] Table 2. The detection performance (Accuracy (TPR / FPR)) under two generative edits applied with different editing strengths t.

[0281] Robustness to Adversarial Watermark Removal. Some recent work demonstrates that adversarial perturbations can be crafted to make watermarks undetectable, even with only black-box access to the detection model. To evaluate robustness against such attacks, the black-box HopSkipJumpAttack (HSJA) is employed, which iteratively crafts minimal perturbations in either the waveform or spectrogram domain to fool the detector. As shown in Table 3, HSJA attacks in both domains with a query budget of Q = 100 are first evaluated. While the attacks successfully reduce detection accuracy, the attacks also significantly degrade audio quality in the process. In the waveform domain, increasing the query budget from Q = 100 to Q = 500 further reduces detection accuracy from 0.68 to 0.47. However, across all experimental configurations, the proposed technique consistently outperforms AudioSeal in terms of both detection robustness and perceptual quality preservation. Aligning with some prior work, the proposed technique is further evaluated against white-box, semi-black-box (using a re-trained surrogate model with the same architecture but different initialization), and Gaussian noise attacks, with results shown in Fig. 5. The results reveal that model vulnerability increases with attacker knowledge, highlighting the importance of keeping detector models proprietary to mitigate watermark removal threats. AudioSeal Propose Technique Diff. Setup

[0282] Ace. PESQ SISNR VisQOL Ace. PESQ SISNR VisQOL Waveform 0.15 1.14 8.97 2.61 0.68 2.80 17.79 3.34 Spectrogram 0.15 1.05 -17.82 2.45 0.36 1.56 -24.08 2.37 #Q=100 0.15 1.14 8.97 2.61 0.68 2.80 17.79 3.34 #Q=200 0.15 1.13 8.95 2.64 0.57 2.30 6.93 2.91 #Q=500 0.15 1.13 8.72 2.63 0.47 1.97 1.66 2.55

[0283]

[0284] Table 3. Detection performance and audio quality metrics after HSJA-based (black-box) adversarial attacks on the waveform and spectrogram domains on Audiomark dataset. Extending to Watermark Localization. As mentioned above, the proposed technique can be further extended to perform watermark localization, for example by following the Brute Force Detection (BFD) method used in WavMark. This is achieved by employing a sliding window detection mechanism. Given that the proposed model operates on 1-second audio segments and incorporates shifting-robust transformations, the per-segment detection probability can be distributed to a finer, per-frame level by utilizing multiple overlapping detection windows. As shown in Fig.6, the proposed technique achieves localization performance comparable to AudioSeal and significantly outperforms WavMark, in terms of both sample-level Fl score and Intersection Over Union (loU) metrics. Furthermore, it is also demonstrated effectiveness in the case where the duration of the watermark segment is fixed at 1 second, and the total audio length varies from 2 seconds to 10 seconds. Please refer to Appendix C.10 for more results.

[0285] Quality and Stealthiness Assessment

[0286] The following objective quality metrics of the watermarked audio are evaluated: Scale Invariant Signal to Noise Ratio (SI-SNR), as well as PESQ, ViSQOL, and STOI.

[0287] Furthermore, it is also reported that the loudness of watermark residual L(5W) according to ITU-R BS.1770-4, which is an important metric for watermarking stealthiness, especially under residual-based detection. As shown in Table 4, the proposed technique achieves competitive performance across all perceptual quality metrics. While WavMark achieves the highest SI-SNR (36.35 dB), the proposed technique excels in perceptual quality measures, achieving the best STOI score (1.000) and the lowest watermark residual loudness (-54.63 dB). For the PESQ and ViSQOL scores, the proposed technique achieves competitive performance, with a PESQ score of 4.43 and a ViSQOL score of 4.56. A subjective listening test is also conducted using the MUSHRA protocol with 12 participants, and the results suggest that the proposed technique achieves a comparable quality with AudioSeal scoring around 91 (while ground truth scores around 95, more details in Appendix C.l and Appendix C.9). In summary, the proposed technique achieves superior detection and attribution performance while maintaining competitive perceptual quality.

[0288] Ablation Study

[0289] It is first conducted ablation on the cross-attention module and temporal modulation layer to showcase their effectiveness for boosting learning in message decoding. As shown in Fig. 8, the proposed approach demonstrates superior learning efficiency, reaching approximately 98% accuracy. When removing the modulation component, performance drops significantly to around 60%. More dramatically, ablating the cross-attention mechanism causes accuracy to drop to random guess levels (50%), indicating this is a crucial architectural element. The proposed ablation studies highlight the contributions of key architectural components.

[0290] Notably, the partial parameter sharing enabled by the cross-attention mechanism is particularly impactful for improving message decoding efficiency and accuracy. While the temporal conditioning module also contributes, the ability of the detector to directly attend to shared message embeddings via cross-attention appears to be a more dominant factor in achieving robust attribution compared to ablations without it. Furthermore, it is ablated the proposed temporal-frequency loss £TFand the adaptive bandwidth with constant weight y = 1. As shown in Table 4, both of them contribute to perceptual quality. Methods SI-SNR $ PESQ $ STOI t ViSQOL $ AudiowMark 27.88 4.59 0.988 4.72 -44.96 Wavmark 36.35 4.43 0.985 4,62 -53.01 TimbreWM 26.45 4.29 0.974 4.63 -43.69 AudioSeal 25.32 4,51 0.990 4.72 -44.51 Proposed Technique 29.00 4.43 1.000 4.56 -54.63 w / o 19.64 4.25 0.990 4.26 -52.63 w / o Adaptive y 21.89 4.22 0.991 4.24 -54.30

[0291]

[0292] Table 4. Audio Quality Metrics. Comparison of objective perceptual quality metrics across watermarking methods.

[0293] Interpretation

[0294] A computing device implementing the techniques described above can have the following example architecture. Other architectures are possible, including architectures with more or fewer components. In some implementations, the example architecture includes one or more processors (e.g., dual-core Intel® Xeon® Processors), one or more output devices (e.g., LCD), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch-sensitive display) and one or more computer-readable mediums (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components can exchange communications and data over one or more communication channels (e.g., buses), which can utilize various hardware and software for facilitating the transfer of data and control signals between components.

[0295] The term “computer-readable medium” refers to a medium that participates in providing instructions to processor for execution, including without limitation, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory) and transmission media.

[0296] Transmission media includes, without limitation, coaxial cables, copper wire and fiber optics. Computer-readable medium can further include operating system (e.g., a Linux® operating system), network communication module, audio interface manager, audio processing manager and live content distributor. Operating system can be multi-user, multiprocessing, multitasking, multithreading, real time, etc. Operating system performs basic tasks, including but not limited to: recognizing input from and providing output to network interfaces and / or devices; keeping track and managing files and directories on computer-readable mediums (e.g., memory or a storage device); controlling peripheral devices; and managing traffic on the one or more communication channels. Network communications module includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols, such as TCP / IP, HTTP, etc.).

[0297] Architecture can be implemented in a parallel processing or peer-to-peer infrastructure or on a single device with one or more processors. Software can include multiple software components or can be a single body of code.

[0298] The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, a browser-based web application, or other unit suitable for use in a computing environment.

[0299] Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto- optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0300] To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or a retina display device for displaying information to the user. The computer can have a touch surface input device (e.g., a touch screen) or a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. The computer can have a voice input device for receiving voice commands from the user.

[0301] The features can be implemented in a computer system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, e.g., a LAN, a WAN, and the computers and networks forming the Internet.

[0302] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client- server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server.

[0303] A system of one or more computers can be configured to perform particular actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0304] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0305] Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0306] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the present disclosure discussions utilizing terms such as “processing”, “computing”, “calculating”, “determining”, “analyzing” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing devices, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0307] Reference throughout this disclosure to “one example embodiment”, “some example embodiments” or “an example embodiment” means that a particular feature, structure or characteristic described in connection with the example embodiment is included in at least one example embodiment of the present disclosure. Thus, appearances of the phrases “in one example embodiment”, “in some example embodiments” or “in an example embodiment” in various places throughout this disclosure are not necessarily all referring to the same example embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more example embodiments.

[0308] As used herein, unless otherwise specified the use of the ordinal adjectives “first”, “second”, “third”, etc., to describe a common object, merely indicates that different instances of like objects are being referred to and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0309] Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms “mounted”, “connected”, “supported”, and “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings.

[0310] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements / features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements / features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.

[0311] It should be appreciated that in the above description of example embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together in a single example embodiment, Fig., or description thereof for the purpose of streamlining the present disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed example embodiment. Thus, the claims following the Description are hereby expressly incorporated into this Description, with each claim standing on its own as a separate example embodiment of this disclosure.

[0312] Furthermore, while some example embodiments described herein include some but not other features included in other example embodiments, combinations of features of different example embodiments are meant to be within the scope of the present disclosure, and form different example embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed example embodiments can be used in any combination.

[0313] In the description provided herein, numerous specific details are set forth. However, it is understood that example embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0314] Thus, while there has been described what are believed to be the best modes of the present disclosure, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the present disclosure, and it is intended to claim all such changes and modifications as fall within the scope of the present disclosure. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present disclosure.

[0315]

[0316] Embodiments

[0317] Various aspects and implementations of the present disclosure may also be appreciated from the following enumerated example embodiments (EEEs), which are not claims.

[0318] EEEA 1. A method for embedding and detecting watermarks in audio signals, comprising:

[0319] a. Decoupling the prediction of watermark bits from the time dimension to reduce computational complexity.

[0320] b. Training a watermarking model that learns embedding tables for message bits and employs a learnable mapping function to compose hidden states.

[0321] c. Enhancing robustness against Al-based editing transformations by focusing on major editing directions in the noise space using sub-space iteration to compute principal components.

[0322] d. Sharing the embedding tables with a detection model designed with a symmetric architecture that matches hidden dimensions.

[0323] e. Training the detection model to detect and reconstruct embedded message bits using cross-attention modules that reference the shared embedding tables.

[0324] EEEA 2. The method of EEEA 1, wherein the learnable mapping function replaces direct averaging in composing hidden message states, thereby improving adaptability and performance in message embedding.

[0325] EEEA 3. The method of EEEA 1, wherein the detection model's cross-attention modules enhance learning efficiency by focusing on relevant embeddings from the watermarking model during message reconstruction.

[0326] EEEA 4. The method outlined in EEEA 1 includes a training component that enhances the robustness of the watermark against Al-based editing transformations by focusing on major editing directions in the noise space. Specifically, during training, the method employs subspace iteration to compute the principal components representing the primary directions of variation introduced by Al-based audio editing models. By incorporating these principal components into the training process, the model learns to embed watermarks that are robust against transformations along these major editing directions.

[0327] EEEA 5. A system for embedding and detecting watermarks in audio signals, comprising: a. A watermarking model configured to embed message bits into audio signals using learned embedding tables and a learnable mapping function, with message prediction decoupled from the time dimension and incorporating robustness enhancement via principal component analysis.

[0328] b. A detection model configured to detect and reconstruct embedded message bits using shared embedding tables and cross-attention modules, designed with a symmetric architecture to the watermarking model.

[0329] EEEA 6. The system of EEEA 5, wherein the symmetric architecture with matching hidden dimensions facilitates efficient inversion of the watermarking process, enhancing detection accuracy.

[0330] EEEA 7. The system of EEEA 5, wherein the decoupling of message prediction from the time dimension in both the watermarking and detection models, reduces computational complexity and accelerates the learning process.

[0331] EEEA 8. The method of EEEA 1, wherein the watermarking and detection models are trained jointly, further improves the efficiency and accuracy of watermark embedding and detection. EEEA 9. The method of EEEA 1, wherein focusing on principal components of Al-based editing transformations, enhances robustness against both traditional processing transformations and Al-based editing transformations in an efficient manner. EEEA 10. A method comprising:

[0332] receiving an original audio signal;

[0333] receiving a message data;

[0334] embedding the message data in the original audio signal using a learned embedding table, thereby generating a watermarked audio signal; and

[0335] causing the watermarked audio signal to be stored or transmitted.

[0336] EEEA 11. The method of EEEA 10, further comprising:

[0337] obtaining a target audio signal; and

[0338] processing the target audio signal using data corresponding to the learned embedding table and cross-attention modules, wherein processing includes:

[0339] determining a decoded candidate message data; and

[0340] determining a prediction data indicating whether the target audio is watermarked. EEEA 12. An electronic device including one or more processors, and a memory including one or more programs including instructions for performing the methods of EEEA 10 or EEEA 11.

[0341] EEEA 13. A non-transitory computer readable medium storing one or more programs including instructions, which when executed by one or more processes of an electronic device, cause the electronic device to perform the methods of EEEA 10 or EEEA 11.

[0342] EEEB 1. A neural network based system for use in audio watermarking, comprising:

[0343] a neural network based watermark generator configured for embedding, based on a learnable embedding table, a predefined watermark into an input audio signal, thereby obtaining a watermarked audio signal; and

[0344] a neural network based watermark detector comprising: a detection head that is configured for determining presence or absence of a watermark in a target audio signal; and a decoding head that is configured for decoding the watermark from the target audio signal, wherein the embedding table is shared between the watermark generator and the watermark detector, such that the decoding of the watermark from the target audio signal is based on the embedding table that has been used for the embedding.

[0345] EEEB 2. The system according to EEEB 1, wherein the watermark comprises a binary message sequence. EEEB 3. The system according to EEEB 1 or 2, wherein the decoding head comprises a cross-attention module configured for decoding the watermark based on the shared embedding table.

[0346] EEEB 4. The system according to any one of the preceding EEEBs, wherein the watermark generator comprises an encoder module configured for transforming the input audio signal into a latent representation for the embedding.

[0347] EEEB 5. The system according to EEEB 4, wherein the watermark generator further comprises a message encoder module configured for mapping the watermark to its latent representation, so as to enable embedding of the latent representation of the watermark into the latent representation of the input audio signal in the latent space, thereby obtaining a modulated latent representation of the input audio signal.

[0348] EEEB 6. The system according to EEEB 5, wherein the latent representation of the watermark is obtained based on a temporal message conditioning mechanism that employs a linear modulation layer.

[0349] EEEB 7. The system according to EEEB 5 or 6, wherein the watermark generator further comprises a decoder module configured for decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained; and wherein the decoder module shares a symmetric structure with the encoder module.

[0350] EEEB 8. The system according to any one of the preceding EEEBs, wherein the watermark detector comprises an encoder module configured for transforming the target audio signal into a reconstructed latent representation.

[0351] EEEB 9. The system according to EEEB 8 when depending on EEEB 3, wherein the cross attention module is configured for decoding the watermark based on the reconstructed latent representation, by referencing the embedding table.

[0352] EEEB 10. The system according to EEEB 9, wherein the embedding table is transformed into a key-value based reference based on linear projection for being used in the referencing. EEEB 11. The system according to EEEB 10, wherein the decoding of the watermark further involves attention-based vector merging with a query from the reconstructed latent representation.

[0353] EEEB 12. The system according to any one of the preceding EEEBs, wherein the neural network based system has been trained end-to-end based on at least one loss function.

[0354] EEEB 13. The system according to EEEB 12, wherein the at least one loss function comprises a perceptual loss function.

[0355] EEEB 14. The system according to EEEB 13, wherein the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency, TF, masking loss based on a psychoacoustic masking principle.

[0356] EEEB 15. The system according to any one of the preceding EEEBs, wherein the watermark detector is further configured to perform watermark localization based on a sliding window detection mechanism.

[0357] EEEB 16. A method of training a neural network based system for use in audio watermarking,

[0358] wherein the neural network based system comprises: a neural network based watermark generator for embedding a watermark into an input audio signal; and a neural network based watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal; and

[0359] wherein the method comprises:

[0360] obtaining, as a training input, a test audio signal and a test watermark sequence that are used as ground truth;

[0361] embedding, by using the watermark generator and based on a learnable embedding table, the test watermark sequence into the test audio signal, thereby obtaining a watermarked audio signal;

[0362] predicting, based on the watermarked audio signal, presence of a watermark by using the detection head, and an estimated watermark sequence by using the decoding head; and iteratively training the neural network based system based on at least one loss function indicative of respective difference between the predicted presence, the estimated watermark sequence, and the ground truth,

[0363] wherein the method further comprises sharing the embedding table between the watermark generator and the watermark detector, such that the prediction of the estimated watermark sequence is based on the embedding table that has been used for the embedding.

[0364] EEEB 17. The method according to EEEB 16, wherein the decoding head comprises a crossattention module; and wherein the estimated watermark sequence is predicted by the crossattention module based on the shared embedding table.

[0365] EEEB 18. The method according to EEEBs 16 or 17, wherein the watermark generator comprises an encoder module; and

[0366] wherein the method further comprises, before the embedding:

[0367] transforming, by using the encoder module, the test audio signal into a latent representation.

[0368] EEEB 19. The method according to EEEB 18, wherein the watermark generator further comprises a message encoder module; and

[0369] wherein the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal involves:

[0370] mapping, by using the message encoder module, the test watermark sequence into its latent representation, so as to enable embedding of the latent representation of the test watermark sequence into the latent representation of the test audio signal in the latent space, thereby obtaining a modulated latent representation of the test audio signal.

[0371] EEEB 20. The method according to EEEB 19, wherein the latent representation of the test watermark sequence is obtained based on a temporal message conditioning mechanism that employs a linear modulation layer.

[0372] EEEB 21. The method according to EEEB 19 or 20, wherein the watermark generator further comprises a decoder module that shares a symmetric structure with the encoder module; and wherein the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal, in order to obtain a watermarked audio signal further comprises:

[0373] decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained.

[0374] EEEB 22. The method according to any one of EEEBs 16 to 21, wherein the watermark detector comprises an encoder module; and

[0375] wherein the method further comprises, before the predicting:

[0376] transforming, by using the encoder module, the watermarked audio signal into a reconstructed latent representation.

[0377] EEEB 23. The method according to EEEB 22 when depending on EEEB 17, wherein the estimated watermark sequence is predicted by the cross attention module based on the reconstructed latent representation, by referencing the embedding table.

[0378] EEEB 24. The method according to EEEB 23, wherein the method further comprises, before the prediction of the estimated watermark sequence, transforming the embedding table into a key-value based reference based on linear projection.

[0379] EEEB 25. The method according to EEEB 24, wherein the prediction of the estimated watermark sequence further involves attention-based vector merging with a query from the reconstructed latent representation.

[0380] EEEB 26. The method according to any one of EEEBs 16 to 25, the at least one loss function comprises a perceptual loss function.

[0381] EEEB 27. The method according to EEEB 26, wherein the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency, TF, masking loss based on a psychoacoustic masking principle.

[0382] EEEB 28. The method according to EEEB 27, wherein the psychoacoustic-aligned TF masking loss involves a weighted L2 loss in the Mel-spectrogram space, using a per-tile weight with asymmetric TF decay. EEEB 29. The method according to EEEB 27 or 28, wherein the psychoacoustic-aligned TF masking loss involves:

[0383] identifying at least one masker region;

[0384] applying TF decay around the masker region, so as to determine a masking threshold; determining at least one maskee region based on the masking threshold; and adjusting weights of the psychoacoustic-aligned TF masking loss, so as to encourage more watermark signal to be embedded in the maskee region.

[0385] EEEB 30. The method according to any one of EEEBs 16 to 29, wherein the method further comprises, before the predicting:

[0386] applying at least one audio transformation to the watermarked audio signal to obtain a transformed audio signal; and

[0387] feeding the transformed audio signal into the watermark detector.

[0388] EEEB 31. The method according to EEEB 30, wherein the audio transformation comprises at least one of: compression, cropping, filtering, boost / duck, echo, noise addition, or artificial intelligence, Al, based technique.

[0389] EEEB 32. The method according to EEEB 31, wherein applying the Al based transformation is based on a set of principal components in noise space obtained by principal component analysis, PCA.

[0390] EEEB 33. The method according to any one of EEEBs 16 to 32, wherein the iteratively training of the neural network based system comprises updating parameters of the watermark generator and / or the watermark detector based on evaluation of the at least one loss function.

[0391] EEEB 34. A method of a neural network based system for audio watermarking, wherein the neural network based system comprises: a watermark generator for embedding a watermark into an input audio signal based on a learnable embedding table; and a watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal, wherein the embedding table is shared between the watermark generator and the watermark detector; and wherein the method comprises:

[0392] obtaining the input audio signal;

[0393] obtaining the watermark; and

[0394] embedding, by using the watermark generator and based on the embedding table, the watermark into the input audio signal, thereby obtaining a watermarked audio signal.

[0395] EEEB 35. The method according to EEEB 34, wherein the method further comprises:

[0396] obtaining the target audio signal;

[0397] determining, by using the detection head, presence or absence of a watermark in the target audio signal; and

[0398] decoding the watermark from the target audio signal by using the decoding head based on the embedding table.

[0399] EEEB 36. A method of a watermark detector for audio watermark detection and decoding, wherein the watermark detector comprises a detection head and a decoding head, and wherein the method comprises:

[0400] obtaining a target audio signal;

[0401] determining, by using the detection head, presence or absence of a watermark in the target audio signal; and

[0402] decoding, by using the decoding head, the watermark from the target audio signal based on a learnable embedding table, wherein the embedding table has been used by a watermark generator during training for embedding a watermark into an input audio signal.

[0403] EEEB 37. A watermark detector, comprising:

[0404] a detection head configured for determining presence or absence of a watermark in a target audio signal; and

[0405] a decoding head configured for decoding the watermark from the target audio signal based on a learnable embedding table, wherein the embedding table has been used by a watermark generator during training for embedding a watermark into an input audio signal.

[0406] EEEB 38. An apparatus, comprising a processor and a memory coupled to the processor, wherein the processor is adapted to carry out the method according to any one of EEEBs 16 to 36. EEEB 39. A program comprising instructions that, when executed by a processor, cause the processor to carry out the method according to any one of EEEBs 16 to 36.

[0407] EEEB 40. A computer-readable storage medium storing the program according to EEEB 39. APPENDIX

[0408] APPENDIX A. Implementation Details

[0409] Model Architecture. Following AudioSeal, a pair of convolutional encoder-decoder models that operate on the waveform domain are leveraged to generate and detect watermarks. The watermark generator consists of a waveform encoder and decoder, both utilizing components from EnCodec. The encoder employs a 1D convolution containing 32 channels and a kernel size of 7, then uses four convolutional blocks. Each of these blocks contains a residual unit and a down-sampling layer, utilizing convolution with stride S and kernel size K = 2S. The residual unit contains two kernel-3 convolutions with a skip-connection, with channels doubling during down-sampling. The encoder ends with a two-layer LSTM and a final 1D convolution having a kernel size of 7 and 128 channels. The stride S values used are (2, 4, 5, 8), and residual units utilize the Exponential Linear Unit (ELU) as a nonlinear activation. The decoder reflects the encoder’s structure but employs transposed convolutions, with strides in the opposite order.

[0410] The detector contains an encoder, transposed convolution, and linear layer for the detection head. The encoder utilizes the same structure as the one from the generator, but with different weights. The transposed convolution includes H output channels and upscales the activation map to match the original audio dimension, yielding an activation map shaped (T, H). This frame-level latent state serves both the detection head and the message decoding head. For the detection head, the linear layer serves to reduce the dimensionality of H to two, followed by a softmax function that produces per-sample detection probability scores. Additional details on the message decoding head are provided in the next section.

[0411] Cross-Attention Message Decoding Head. The message decoding head aims to decode the embedded message bits by leveraging the shared embedding table from the generator. The key idea is to have a “message vocabulary” to look up when decoding the message bits. To this end, it is employed a single-head cross-attention layer with the message latent state as a query and the units in the embedding table as key- value pairs. Note that the reconstructed waveform latent is first passed through a fully connected layer to decompose the time dimension and obtain a K-dimensional message latent state as a query. For the embedding table part, the embedding table is reshaped to a 2H-dimensional vector by combining the two vicinal units that correspond to the same message bit into one 2H-dimensional vector. All the linear projections Q, K, V are designed to map the latent dimension to H dimensions. After the cross-attention operation, an ELU activation function is applied, followed by a final linear projection W E KHxlto obtain the final per-bit message logits for the logistic regression. Training and Inference Details. During the training stage, a curriculum learning method is employed to first disable the gradient-based quality balancing term, guiding the model to focus more on learning watermark decoding at the beginning of training. After 59000 training steps, the quality balancing term is enabled to help the model improve the watermark auditory transparency with more dynamic scaling across different perceptual loss terms. To boost the sampling of the transformation efficiency, the sampling probability of each transformation is updated every 1000 steps on the validation set, adjusting it based on the validation accuracy of each transformation. In detail, the sampling ratio pgfor transformation

[0412]

[0413] g is given by pg= accg / + e, where accgis the accuracy of the model on transformation

[0414]

[0415] 1-accg

[0416] g and e is a small constant for keeping the sampling ratio non-zero. It is observed that it boosts the learning efficiency in attribution with at least 2x improvement under the main setup. The training checkpoint at 73000 is selected based on its balanced performance between quality and utility scores on the validation set. During the inference stage, for given audio with arbitrary duration, the input audio is first padded and split into multiple Is audio segments and the watermarking model is then applied to each chunk. Audio chunks are then concatenated to obtain the final watermarked audio.

[0417] Perceptual Loss. Following AudioSeal, a pool of perceptual loss terms are employed to guide the model to learn the watermarking task. The perceptual loss terms include: i) multiscale mel-spectrogram loss Tmsspecthat computes both LI loss on linear-scale mel-spectrograms and MSE loss on log-scale mel-spectrograms across multiple FFT window sizes (from 26to 211), with each scale weighted by 2i— 1 for scale i; ii) feature matching loss £featthat minimizes LI distance between intermediate feature maps extracted from the adversarial discriminator, averaged across all layers; iii) waveform LI loss Twaveformthat minimizes LI distance between the original waveform and the watermarked waveform; iv) the proposed psychoacoustic-aligned temporal-frequency masking loss Lmask, which captures the weighted

[0418]

[0419] loss in the mel-spectrograms domain, using a per-tile weight obtained through simulation of the masker energy decaying in the time-frequency domain. For the adversarial discriminator, a multi-scale STFT discriminator architecture is adopted that operates on the complex STFT representations of the audio at different scales.

[0420] Specifically, it consists of multiple sub-discriminators, each processing the STFT with different FFT sizes (512, 1024, 2048), hop lengths (128, 256, 512), and window sizes (512, 1024, 2048). Each sub-discriminator first computes the STFT, concatenates the real and imaginary components along the channel dimension, and processes them through a series of 2D convolutions with increasing dilation rates (1,2,4). The feature maps from these convolutional layers are used for the feature matching loss. These losses jointly optimize for perceptual quality by matching both time-frequency characteristics and learned audio representations.

[0421] Evaluation Setup. For the attribution experiment, following San Roman et al., a pool of N potential users are set up, where each user’s sequential bit message is derived from their ordinal number, i.e., the i-th user’s message is given by wi= binary(i). This creates a space for all potential messages Ew. The attribution is achieved by retrieving the message in Ewwith the smallest Hamming distance. Compared to bit-wise message accuracy, this metric provides a more practical measure of message retrieval performance, aligning better with real-world watermarking applications and attribution use cases. The message length (in bits) determines the maximum number of uniquely identifiable users. Specifically, for a message of K bits, the system can support up to 2Kunique users. In the experiments, a 16-bit message is used, enabling attribution across 216= 65,536 users at maximum.

[0422] A.l. Robustness Augmentations

[0423] Following AudioSeal, these audio editing augmentations are applied during training and evaluation:

[0424] • Bandpass Filter: Simulates frequency-selective audio equipment by allowing only mid-range frequencies. Allows frequencies between 300Hz and 8000Hz to pass through.

[0425] • Highpass Filter: Removes bass frequencies to simulate poor bass response. Cuts frequencies below 500Hz.

[0426] • Lowpass Filter: Removes high frequencies to simulate muffled audio. Cuts frequencies above 5000Hz.

[0427] Speed: Simulates playback speed variations. Changes speed by random factor 0.8- 1.2. • Resample: Robustness to sample rate conversion. Upsamples to 32kHz, then downsamples back to the original rate.

[0428] • Boost Audio: Simulates volume increase. Amplifies by factor 1.2.

[0429] • Duck Audio: Simulates volume decrease. Reduces volume by a factor of 0.8.

[0430] • Echo: Simulates room acoustics and reverberation. Adds delayed copy with 0.1-0.5s delay and 0.1-0.5 volume.

[0431] • Pink Noise: Adds realistic environmental noise. Adds pink noise with std 0.01.

[0432] • White Noise: Adds random electronic noise. Adds Gaussian noise with std 0.001.

[0433] • Smooth: Simulates low-quality audio processing. Moving average filter with window size 2-10.

[0434] • AAC: Robustness to common lossy compression. AAC encoding at 128kbps.

[0435] • MP3: Robustness to common lossy compression. MP3 encoding at 128kbps.

[0436] • EnCodec: Tests neural audio codec compression. Resamples to 24kHz, encodes with 16 streams (nq= 16), resamples to 16kHz.

[0437] • Crop: Robustness to audio truncation, padding, and in-batch audio mixing. While San Roman et al. use this to obtain a localization mask, the present disclosure implements crop as one of the edits to gain cropping-based robustness. Specifically, k starting points are first randomly selected, T / 2k consecutive samples are then modified in one of four ways:

[0438] - Revert to original audio (40% probability)

[0439] - Replace with zeros (20% probability)

[0440] Substitute with different audio from the same batch (20% probability) - Leave unmodified (20% probability)

[0441] A.2. Dataset Details

[0442] The present disclosure utilizes a mixed dataset of 4100 hours in total for training, which contains 100.59-hour LibriSpeech, 98.53-hour AudioSet, 879.29-hour Free Music Archive, 9-hour MusicCaps, and 3,016.43-hour VoxPopuli. Each dataset is described as follows: LibriSpeech. LibriSpeech is an English speech dataset derived from audiobooks in the LibriVox project. The dataset contains approximately 1000 hours of read English speech sampled at 16kHz, with careful segmentation and alignment. The audio is paired with transcribed text, making it suitable for speech recognition tasks. A 100.59-hour subset of the full 1000-hour dataset is used for training.

[0443] AudioSet. AudioSet is a large-scale dataset containing 2,084,320 human-labeled 10-second sound clips drawn from YouTube videos. It consists of 632 audio event classes organized in a hierarchical ontology, covering a wide range of sounds, including human and animal sounds, musical instruments, genres, and common environmental sounds. The dataset contains approximately 5,790 hours of annotated audio. For the experiments, a 98.53-hour randomly sampled subset of the full dataset is used for training.

[0444] Free Music Archive. FMA is a large-scale, open-source dataset of music tracks with clear licensing. It uses the “large” subset containing 879.29 hours of audio data with 106,57430-second tracks sampled at 44.1kHz. The dataset spans multiple genres, including Rock, Electronic, Experimental, Hip-Hop, Folk, and Instrumental, making it suitable for training robust watermarking models across diverse musical styles.

[0445] MusicCaps. MusicCaps is a dataset of 5.5k music-text pairs (9.47 hours total), with high-quality human-written captions describing musical attributes like genre, mood, instruments, and tempo. The dataset was created by having music experts write detailed captions for a subset of 5.5k music clips from AudioSet, with each caption carefully describing the musical content, instrumentation, style, and other sonic characteristics. It uses a subset of 9 hours for training.

[0446] VoxPopuli. VoxPopuli is a large-scale multilingual speech corpus collected from European Parliament event recordings between 2009-2020. It contains approximately 400K hours of unlabeled speech data across 23 languages, 1.8K hours of transcribed speech for 16 languages, and 17.3K hours of speech-to-speech interpretation data. For English specifically, it provides 24. IK hours of unlabeled speech data and 543 hours of transcribed speech from 1,313 speakers. The dataset has been widely used for representation learning, semisupervised learning, and interpretation tasks. It uses a 3,016.43-hour subset that covers balanced speech from 23 languages for training. APPENDIX B: Discussion on Limitations

[0447] While the proposed method demonstrates strong performance across various standard audio transformations, several limitations and opportunities for improvement remain. First, although the present disclosure achieves robust performance against most common audio edits, the attribution accuracy degrades significantly for challenging operations like speed changes and generative editing. Specifically, as shown in Table 7, the proposed technique achieves only 4% average attribution accuracy for speed changes across different user setups, though this still represents an improvement over existing baselines which fail completely. To enhance robustness against speed variations, one promising direction is to incorporate a temporal adjustment layer similar to Westerfeld, which performs black-box optimization to identify and reverse speed changes. It is presented preliminary work in this direction in Appendix C.3.1. Additionally, the current solution’s attribution robustness against Al-based editing remains an area requiring further improvement.

[0448] APPENDIX C: More Results

[0449] C.l. Subjective Evaluation with MUSHRA

[0450] The MUSHRA protocol is a crowdsourced test in which participants rate the quality of various samples on a scale of 0 to 100. The ground truth is provided for reference. The present disclosure utilized eight randomly sampled speech and music samples from different sources, each lasting 5 seconds. As part thereof, the present disclosure included a low anchor, which is a low-pass-filtered version at 3.5kHz. Each sample was evaluated by 16 participants, with a post-hoc filtering process to exclude participants who failed to correctly identify more than two hidden reference samples in their scoring, resulting in 12 valid participants at the end. For comparison, the ground truth samples received an average score of around 98, while the low anchor’s average score was around 25. As shown in Fig. 9, the proposed method achieved a score of approximately 91, closely matching AudioSeal. Among all the methods, Audiowmark is the one that achieves the highest score at around 94. WavMark received a slightly lower score of 88, while Timbre WM scored the lowest, around 80. These results demonstrate that the proposed watermarking approach maintains a perceptual quality that is competitive with state-of-the-art methods. C.2. Analysis on the Training Dynamics of Models with Different Architectures The present disclosure analyzes the training dynamics of different architectures under a controlled experimental setup to better understand their inherent learning capabilities. To isolate architectural effects, the proposed techniques remove most auxiliary losses, retaining only the adversarial loss (weight=l) for all methods. Additionally, the present disclosure includes a waveform-domain

[0451]

[0452] loss (weight=l) for AudioSeal and the proposed method, as it was observed that it improves the learning efficiency. Fig. 10 illustrates the validation accuracy trajectories and quality for both watermark detection and message-bit decoding during training. The results reveal distinct learning characteristics that highlight the trade-offs between different architectural choices.

[0453] WavMark’s fully-shared architecture exhibits remarkably rapid initial learning, converging to high accuracy within just a few hundred steps. However, this early success is followed by a concerning degradation in detection accuracy as training progresses. This pattern suggests that while full parameter sharing enables quick initial learning, it may lead to destructive interference between the detection and the decoding tasks during extended training.

[0454] AudioSeal’s fully-disjoint architecture in general shows slow convergence in both detection and message-bit decoding. After training for 30k steps, it quickly converges to 99% detection accuracy but the learning of message-bit decoding progresses slowly, achieving around 70% message bit accuracy at the end of 50k training steps. Moreover, the perceptual quality of AudioSeal is also degraded more than the proposed technique of the present disclosure and WavMark.

[0455] Compared to these, the proposed blended architecture demonstrates steady and comprehensive learning - both detection and decoding accuracies improve gradually, ultimately achieving near-perfect performance only with about 19k training steps. Notably, with only 2k training steps, the proposed technique can converge to 99% detection accuracy. This great boost in learning efficiency suggests that the proposed partial parameter- sharing strategy and cross-attention mechanism enable more efficient learning of both tasks.

[0456] C.3. Robustness to Standard Edits

[0457] The present disclosure presents the full results summarizing detection and attribution robustness against standard edits in Tables 6 and 7. For detection, the proposed method achieves consistently high performance across all transformations, with an average accuracy of 99.19% and AUC of 99.93%, significantly outperforming all baselines. Even for challenging transformations like speed changes and neural codec compression (EnCodec) where other methods mostly fail (accuracy around 50-60%), the proposed approach maintains exceptional detection performance with accuracies of 99.5% and 96.5%, respectively.

[0458] For attribution, the results in Table 7 show that the proposed method maintains high attribution accuracy (92-94% average) across different user pool numbers (and associated messages), demonstrating strong scalability. The performance is particularly robust for most standard audio transformations like bandpass filtering, boost / duck, echo, and noise addition, achieving near 100% attribution accuracy. While speed changes remain challenging, the proposed method still achieves better performance than baselines which completely fail. This indicates that while the proposed method handles most common audio edits exceptionally well, speed changes continue to be a challenging transformation for reliable attribution. The present disclosure also explores a potential simple extension of speed reversion to further boost the performance in Appendix C.3.1.

[0459] The proposed approach shows particularly strong robustness against modem audio codecs -maintaining near-perfect attribution accuracy for both MP3 and AAC compression (98-100%), while baselines like AudioSeal struggle significantly with these transformations (showing accuracies below 60%). This demonstrates the proposed method’s effectiveness against state-of-the-art audio compression techniques that are increasingly common in real-world applications. Additionally, the proposed method shows outstanding robustness to cropping operations, maintaining 100% attribution accuracy, while AudioSeal struggles in this task (12% accuracy).

[0460] To further clarify the proposed attribution evaluation, particularly regarding the metrics used, the present disclosure also reports the False Attribution Rate (FAR) and compares different definitions of attribution accuracy, aligning with methodologies in AudioSeal. In the present work, attribution accuracy is defined as the fraction of correctly attributed samples among those successfully detected as watermarked. This metric is equivalent to 1 - FAR (False Attribution Rate, as reported in AudioSeal). The present disclosure adopts this definition because it allows for a decoupled analysis of pure attribution performance from the initial detection performance. In AudioSeal, the attribution accuracy is defined as the fraction of correctly attributed samples among all samples, which couples the performance of both detection and attribution. The present disclosure presents the results on both the MusicCaps and VoxPopuli datasets in Table 5, illustrating performance under these specific metrics. The results suggest that in speech domain, when the user pool size is relatively small, AudioSeal is slightly better than the proposed technique of the present disclosure, and in all the other setups, the proposed technique consistently outperforms AudioSeal and WavMark. Moreover, as indicated above, despite AudioSeal achieving robust detection, the model struggles to perform accurate message decoding even without distortion (See Table 6).

[0461] Dataset Metric Method N=10 N=100 N=1000 N= 10000

[0462] WavMark 14.67 14.92 15.18 16.07 AudioSeal 18.77 25.76 36.63 30.53 FAR (%);

[0463] Proposed 6.40 6.13 6.44 6.56 Technique

[0464] WavMark 85.33 85.08 84,82 83.93 AudioSeal 81.23 74.24 63.37 69.47 MusicCaps Acc. (Att.) (%)$

[0465] Proposed 93.60 93.87 93.56 93.44 Technique

[0466] WavMark 79.69 80.56 79.94 79.06 Acc. (Det. + Att.) AudioSeal 80.81 73.56 62.69 68.75 (%)t Proposed 90.50 87.19 90.75 91.81

[0467] Technique

[0468] WavMark 13.11 13.21 13.96 15.00 AudioSeal 3.71 12,86 18.91 16.31 FAR (%);

[0469] Proposed 5.03 6.01 6.25 7.70 Technique

[0470] WavMark 86.89 86.79 86.04 85.00 AudioSeal 96.29 87.14 81.09 83.69 VoxPopuli Acc. (Att.) (%)$

[0471] Proposed 94,97 93.99 93.75 92.30 Technique

[0472] WavMark 78.69 78.34 78.06 76.88 Acc. (Det. + Att.) AudioSeal 89.50 86.60 79.83 83.45 (%)t Proposed 92.45 92.69 92.49 91.08

[0473] Technique

[0474]

[0475] Table 5. Attribution results over edits on MusicCaps (the proposed main setup) and VoxPopuli (align with AudioSeal’s setup in San Roman et al.

[0476] The present disclosure reports the false attribution rate (FAR) and two attribution accuracy metrics (Att. and Det.+Att.) The key difference between the proposed definition of attribution accuracy and AudioSeal’s is that the attribution performance is isolated from the detection performance. The best score is in bold, and the runner-up is underlined.

[0477] AudiowMark WavMark TimbreWM AudioSeal Propose Technique Acc. AUC Acc. AUC Acc. AUC Acc. AUC Acc. AUC Edit (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) Identity 1.00 1.00 1.00 1.00 0.995 1.00 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.99 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Bandpass 1.00 1.00 1.00 1.00 0.985 1.00 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Boost 1.00 1.00 1.00 1.00 0.98 (0.96 / 0.00) 0.9998 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.99 / 0.00) (0.99 / 0.00) Duck 1.00 1.00 0.995 0.995 0.97 (0.97 / 0.00) 0.9999 1.00 1.00 0.995 1.00 (1.00 / 0.0) (0.99 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Echo 1.00 1.00 1.00 1.00 0.945 0.9965 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.95 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Highpass 1.00 1.00 0.95 0.95 0.985 1.00 1.00 1.00 0.995 1.00 (1.00 / 0.0) (0.90 / 0.00) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Lowpass 1.00 1.00 1.00 1.00 0.985 0.9986 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.97 / 0.00) (0.99 / 0.00) (0.99 / 0.00) MP3 0.94 0.94 0.805 0.805 0.95 (0.91 / 0.01) 0.9971 1.00 1.00 0.995 1.00 (0.88 / 0.00) (0.61 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Pink 1.00 1.00 1.00 1.00 0.995 1.00 1.00 1.00 0.995 1.00 Noise (1.00 / 0.0) (1.00 / 0.0) (0.98 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Random 1.00 1.00 1.00 1.00 0.975 0.9999 1.00 1.00 0.995 1.00 Noise (1.00 / 0.0) (1.00 / 0.0) (0.95 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Smooth 1.00 1.00 0.96 0.96 0.98 (0.96 / 0.00) 0.9976 1.00 1.00 0.995 1.00 (1.00 / 0.0) (0.92 / 0.00) (0.99 / 0.00) (0.99 / 0.00) Speed 0.50 0.50 0.50 0.50 0.515 0.4812 0.61 0.61 0.995 1.00 (0.00 / 0.0) (0.00 / 0.0) (0.07 / 0.04) (0.36 / 0.15) (0.99 / 0.00) Resample 1.00 1.00 1.00 1.00 0.98 (0.96 / 0.00) 1.00 1.00 1.00 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.99 / 0.00) (0.99 / 0.00) AAC 1.00 1.00 1.00 1.00 0.975 0.9999 1.00 1.100 0.995 1.00 (1.00 / 0.0) (1.00 / 0.0) (0.95 / 0.00) (0.99 / 0.00) (0.99 / 0.00) EnCodec 0.50 0.50 0.805 0.805 0.625 0.7092 1.00 1.00 0.965 0.991 (nq=16) (0.00 / 0.0) (0.61 / 0.00) (0.50 / 0.25) (0.99 / 0.00) (0.93 / 0.00)

[0478] Crop 0.965 0.965 0.995 0.995 0.96 (0.92 / 0.00) 0.9918 1.00 1.00 0.975 0.997 (0.93 / 0.00) (0.99 / 0.00) (0.99 / 0.00) (0.98 / 0.03) Average 0.9294 0.9294 0.9181 0.9181 0.925 0.9482 0.9706 0.9757 0.9919 0.9993 (0.8588 / 0.00) (0.8363 / 0.00) (0.8688 / 0.0188) (0.95 / 0.01) (0.9858 / 0.0019)

[0479]

[0480] Table 6. The full detection results of the present technique across different standard audio edits on the MusicCaps dataset. Acc. (TPR / FPR) is the accuracy (and TPR / FPR) obtained for the optimal threshold from Youden’s Index on a balanced validation set. AudiowMark Wavmark Timbre WM AudioSeal Proposed Technique Transformation

[0481] N=100 1000 10000 Avg 100 1000 10000 Avg 100 1000 10000 Avg 100 1000 10000 Avg 100 1000 10000 Avg Identity 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.93 0.86 0.93 0.75 0.50 0.50 0.69 1.00 1.00 1.00 1.0 Encodec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.07 0.50 0.00 0.00 0.31 1.00 0.99 0.99 0.9 AAC 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.94 0.90 0.85 0.90 0.50 0.00 0.00 0.12 0.99 0.96 0.69 0.8 Bandpass 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.92 0.92 0.89 0.93 0.50 0.25 0.00 0.31 0.99 0.99 0.99 0.9 Boost 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.91 0.91 0.87 0.91 0.25 0.50 0.25 0.50 1.00 1.00 1.00 1.0 Duck 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.93 0.85 0.84 0.89 0.50 0.50 0.25 0.56 1.00 1.00 1.00 1.0 Echo 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.92 0.92 0.87 0.91 0.25 0.25 0.25 0.38 1.00 0.98 1.00 0.9 Highpass 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.92 0.87 0.93 0.50 0.25 0.00 0.31 1.00 1.00 1.00 1.0 Lowpass 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.93 0.86 0.85 0.90 0.50 0.50 0.50 0.56 1.00 1.00 1.00 1.0 MP3 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.85 0.85 0.86 0.86 0.50 0.50 0.00 0.38 1.00 1.00 1.00 1.0 Pink Noise 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.93 0.91 0.87 0.92 0.75 0.75 0.75 0.75 1.00 1.00 1.00 1.0 White Noise 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.88 0.89 0.87 0.90 0.50 0.75 0.50 0.56 1.00 1.00 1.00 1.0 Smooth 1.00 1.00 1.00 1.00 1.00 1.00 0.75 0.88 0.75 0.86 0.69 0.80 0.00 0.25 0.25 0.19 1.00 1.00 0.99 1.0 Speed 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.04 0.19 0.18 0.00 0.00 0.00 0.00 0.00 0.00 0.08 0.0 Resample 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.93 0.92 0.87 0.92 0.75 0.25 0.25 0.56 1.00 1.00 1.00 1.0 Crop 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.93 0.76 0.80 0.85 0.00 0.00 0.00 0.12 1.00 0.99 1.00 1.0 Avg 0.88 0.88 0.88 0.88 0.88 0.88 0.86 0.87 0.81 0.78 0.75 0.80 0.42 0.33 0.22 0.39 0.94 0.93 0.92 0.9

[0482]

[0483] Table 7. The full attribution results for all methods, and associated overall average performance, across all transformations on the MusicCaps dataset C.3. I. SIMPLE SPEED REVERSION LAYER WITH BLACK-BOX GRID SEARCH

[0484] The present disclosure explores a post-hoc simple speed reversion layer with a black-box grid search to further boost the robustness against speed changes. The speed reversion layer is added on top of the watermarking model. The main idea is to reverse the speed change by searching for the optimal speed factor with the guidance of the watermarking model. It is observed that on the unwatermarked audio, changing the speed cannot adversarially increase the detection accuracy, while on the watermarked audio, speed changing will significantly affect the watermark message decoding pattern. It is observed that the watermarking model predicts more stable message bits for each 1-second chunk when the speed remains unchanged. However, after applying a speed change, the model predicts more diverse message bits with lower confidence for each chunk. To leverage this insight, given the original speed change space [ymin, Tmax] and the subsequent search space l~ —, the

[0485]

[0486] L / max / min-l present disclosure designs the score function r(a) as the average of the watermarking model’s mean detection score over each chunk p(a), and the average standard deviation of the predicted message bits sm^. Formally, this is expressed as r(a) = ^f=1(p(a) +

[0487]

[0488] where L is the number of chunks, a is the speed factor, and m(a) is the predicted message bits. Then it is conducted a two-round linear grid search to find the optimal speed factor a that maximizes the score function r(a), with the first round step size as S1?and the second round step size as S2. The speed change parameter yminand ymaxare set to 0.8 and 1.25, respectively. Experiments are conducted on both watermarked and unwatermarked audio to investigate whether speed reversion might cause false positives. For the watermarked case, it is reported the attribution success rate / detection time, and for the unwatermarked case, it is reported the FPR. As shown by the results in Table 8, with more fine-grained search space, the speed reversion layer can further improve the robustness against speed changes while maintaining the overall detection efficiency. Meanwhile, for unwatermarked audio, applying the speed reversion layer does not falsely increase the probability of being classified as watermarked by the detection model.

[0489] w / o Search w / Search = 0.1, S2= w / Search = 0.01, S2= 0.01) 0.001)

[0490] #User

[0491] Success Det. Success Det. FPR Success Det. FPR Rate Time (s) Rate Time (s) Rate Time (s) N=2 0.391 0.005 0.537 0.220 0.01 1.000 0.653 0.00

[0492]

[0493] N=10 0.069 0.006 0.237 0.229 0.00 1.000 0.583 0.01 N=100 0.222 0.006 0.612 0.230 0.00 0.990 0.545 0.00 N=1000 0.000 0.007 0.163 0.230 0.00 1.000 0.497 0.00

[0494]

[0495] Table 8. Comparison of speed reversion layer performance.

[0496] Both the success rate and the average detection time (in seconds) are reported for different user pool sizes N. The speed reversion layer significantly improves robustness against speed changes while maintaining efficient detection time.

[0497] C.4. Robustness to Generative Edits

[0498] The present disclosure uses the DDIM-inversion-based editing approach of ZETA (Text-Based Audio Editing) introduced in Manor & Michaeli to manipulate the audio with text prompts. The present disclosure briefly summarizes the ZETA editing process in the next section. For more details on this editing process, please refer to Manor & Michaeli.

[0499] ZETA. Given the original signal x0(waveform or latent space), this approach first conducts guided forward process xt= jlTtxQ

[0500]

[0501] + 1 — atet, t = 1,, T, where atis the cumulative variance schedule, and etis the independent Gaussian noise. Then, a sequence of noise vectors that capture the main characteristics of the source audio is extracted using zt= xt-1— |it(t)) / crt, t = T,...,1, where |it(-) is a denoising neural network trained to predict the mean component of the noise added at time t. Note that a source prompt psrccan also be used at the conditioning layer of the denoising neural network to guide the forward diffusion process. During the editing process, given a target prompt ptgtand a noisy state xT, a guided diffusion backward process is conducted via xt-1= ^(xt) + atzt, t = T,...,1, where the noise vector sequence ztare the ones extracted from the previous forward process. The target prompt ptgtis embedded and applied to the cross-attention part that affects the editing trajectory.

[0502] The present disclosure presents the full results of robustness against generative edits in Table 9, where the proposed method is evaluated against two state-of-the-art audio generation models: Stable Audio and AudioLDM2. Different forward diffusion steps T are used to create varying strengths of audio manipulation, ranging from 10 to 110. The source prompt psrcis set to “A recording of music”, and the target prompt ptgtis set to “A recording of EDM music with strong rhythm”. The original default settings are followed for other

[0503] hyperparameters. For Stable Audio, the proposed method achieves consistently high accuracy (0.81-0.94) and AUC scores (0.89-0.99) across different T values. In contrast, AudioSeal shows more variable performance: while it achieves decent accuracy (0.69-0.78) for lower T values (10-30), its performance degrades significantly as T increases, with accuracy dropping to around 0.59 and AUC falling to around 0.50-0.53 for higher T values. For AudioLDM2, the proposed approach maintains strong performance with an accuracy of around 0.94 and an AUC above 0.93 across all diffusion steps. AudioSeal again struggles with higher T values, showing consistent accuracy around 0.59 but poor AUC scores (0.33-0.53).

[0504] Furthermore, it is also compared against Timbre Watermark, a frequency-based watermarking method that has demonstrated state-of-the-art robustness against voice cloning training pipelines. While this approach could potentially handle generative edits, which also modify audio significantly, the experiments show that it performs poorly on this task. As shown in Table 9, Timbre Watermark achieves only random-guess level performance (accuracy around 0.50) for both Stable Audio and AudioLDM2 across different T values, significantly underperforming both AudioSeal and the proposed method.

[0505] Editing AudioSeal Timbre W atermar k Proposed Technique T

[0506] Model Acc. (TPR / FPR) AUC Acc. (TPR / FPR) AUC Acc. (TPR / FPR) AUC 10 0.6875 0.7227 0.4688 0.4219 0.9375 0.9883 (0.625 / 0.25) (0.8125 / 0.875) (0.9375 / 0.0625)

[0507] 30 0.7813 0.8125 0.4063 0.3789 0.8125 0.8906 (0.875 / 0.3125) (0.5625 / 0.75) (0.75 / 0.125)

[0508] 50 0.5938 0.3398 0.4688 0.4863 0.9063 0.9688 (0.875 / 0.75) (0.9625 / 0.125) (0.875 / 0.0625) Stable Audio

[0509] 70 0.5938 0.5313 0.5313 0.5547 0.9063 0.9648 (0.75 / 0.5625) (0.1875 / 0.125) (0.875 / 0.0625)

[0510] 90 0.5938 0.5469 0.5000 0.5996 0.9063 0.9688 (0.875 / 0.6875) (0.5000 / 0.5000) (0.875 / 0.0625)

[0511] 110 0.5938 0.5039 0.5313 0.5488 0.9063 0.9570 (0.9375 / 0.75) (0.5625 / 0.5000) (0.875 / 0.0625)

[0512] 10 0.5938 0.5273 0.5313 0.4961 0.9375 0.9492 (0.9375 / 0.75) (0.8125 / 0.75) (0.9375 / 0.0625)

[0513] 30 0.5938 0.3945 0.4375 0.4570 0.9375 0.9414 AudioLDM2

[0514] (0.9375 / 0.75) (0.0 / 0.125) (0.9375 / 0.0625)

[0515] 50 0.56 (0.88 / 0.75) 0.34 0.5313 0.5645 0.9063 0.9336

[0516] (0.5625 / 0.5000) (0.875 / 0.0625)

[0517]

[0518] 70 0.5938 0.3477 0.5000 0.5703 0.9375 0.9453 (0.9375 / 0.75) (0.125 / 0.125) (0.9375 / 0.0625)

[0519] 90 0.5938 0.3945 0.5000 0.5723 0.9375 0.9414 (0.9375 / 0.75) (0.5000 / 0.5000) (0.9375 / 0.0625)

[0520] 110 0.5938 0.3359 0.5000 (0.0 / 0.0) 0.5137 0.9375 0.9414 (0.9375 / 0.75) (0.9375 / 0.0625)

[0521]

[0522] Table 9.

[0523] It is demonstrated that the proposed watermarking approach can generalize better to the unseen, challenging generative edits setting. Detection Results for two generative edits applied with different forward t steps.

[0524] C.5. Generalization Across Standard Editing with Different Strength

[0525] The generalization of the proposed technique against transformations of different strengths is represented in Fig. 11. The proposed method demonstrates remarkable robustness across a wide spectrum of audio transformations with varying intensities. For lossy compression operations, EnCodec shows detection accuracy of 69.5-95.0% and attribution rates of 70.0-100% depending on bitrate. AAC maintains strong detection (97.5%) across all bitrates with attribution ranging from 53.0-98.0%. MP3 compression achieves consistent detection (97.5%) and perfect attribution (100%) across all bitrates.

[0526] For frequency-domain operations, the proposed watermark remains highly effective. When filtering out low frequencies (highpass filtering), the proposed technique maintains excellent performance with 99.0-99.5% detection accuracy and 99.0-100% attribution accuracy, even as the cutoff frequency is varied from 500 Hz to 4000 Hz. Similarly, when filtering out high frequencies (lowpass filtering with cutoffs between 500-2500 Hz), the proposed technique achieves 97.5% detection accuracy while maintaining 100% attribution accuracy. For volume changes, the proposed method handles both increases and reductions well - maintaining 97.5% detection and 100% attribution for volume boosts up to 90%, and only showing slight degradation (85.0% detection) when reducing volume to 10% of original while still maintaining 100% attribution.

[0527] For additive noise, both Gaussian and pink noise maintain 97.5% detection accuracy up to 0.1 standard deviations, though attribution degrades from 100% at low noise levels (<0.01) to 0% at high intensities (>0.5). Speed changes achieve 94.5-97.5% detection but, as expected, only maintain attribution at normal speed (1. Ox). For resampling operations across frequencies from 16 kHz to 1024 kHz, detection remains strong (97.5%) with perfect attribution (100%). Echo effects across different delay times (0.1-0.5) consistently achieve 97.5% detection and 100% attribution, demonstrating the watermark’s comprehensive resilience to common audio transformations.

[0528] C.6. Generalization Across Different Datasets

[0529] To study how the proposed technique generalizes across different domains and datasets, indistribution and out-of-distribution tests are conducted on multiple audio datasets. 100 audio clips with 5s duration are randomly sampled from each dataset. The user size N is set to 1000.

[0530] For in-distribution evaluation (Table 10), it is tested on FMA-Large (music), EibriSpeech (speech), VoxPopuli-OK (speech), and AudioSet (general audio). The proposed method achieves consistently high detection accuracy (96-99%) and attribution accuracy (87-93%) across all datasets and editing operations. The only exception is speed change, where attribution fails as expected since it fundamentally alters the temporal structure that watermarks rely on. It is also observed slightly lower performance on EnCodec compression for FMA-Large and AudioSet, where detection accuracy drops to 72-81% and attribution accuracy to 70-97%, reflecting the challenge of handling highly compressed audio. Notably, the average detection accuracy across all datasets remains strong at 96-99%, with attribution accuracy ranging from 87-93%.

[0531] For out-of-distribution generalization (Table 11), it is evaluated on four unseen datasets: AudioMarkBench (which is the speech subset from Common Voice), ASVspoof (speech), MusicGen (Al-generated music), and MUsDB18 (music). The proposed method maintains robust performance on these challenging OOD datasets, with detection accuracy ranging from 93-99% and attribution accuracy from 86-94% The speech datasets (AudioMarkBench and ASVspoof) show slightly lower but still strong detection rates of 93-97%, while music datasets (MusicGen and MUsDB18) achieve near-perfect detection at 98-99%. Attribution performance remains consistently high across all OOD datasets, except for speed changes, which consistently fail as expected. Notably, the average attribution accuracy for OOD datasets ranges from 86-94%, demonstrating the robustness of the proposed watermarking approach across diverse audio domains.

[0532] These results highlight the strong generalization capabilities of the proposed method, both indistribution and out-of-distribution. The consistent performance across datasets and editing operations, with only expected failures (e.g., speed changes and extreme compression), underscores the practical applicability of the proposed watermarking system in real-world scenarios where diverse audio content and editing operations are encountered.

[0533] Dataset FMA-Large LibriSpeech VoxPopuli-10K AudioSet Edit Det. Att. Det. Att. Det. Att. Det. Att.

[0534] (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) Identity 0.995 1.000 0.995 1.000 0.997 1.000 0.990 0.970

[0535] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.980 / 0.000) Bandpass 0.995 0.980 0.995 1.000 0.997 1.000 0.995 0.990

[0536] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.000) Boost 0.995 0.990 0.995 1.000 0.997 1.000 0.985 0.970

[0537] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.980 / 0.010) Duck 0.995 1.000 0.995 1.000 0.997 1.000 0.995 0.980

[0538] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.000) Echo 0.990 1.000 0.995 1.000 0.997 1.000 0.985 0.960

[0539] (0.980 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.970 / 0.000) Highpass 0.995 0.990 0.995 1.000 0.997 1.000 0.995 1.000

[0540] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.000) Lowpass 0.965 1.000 0.995 1.000 0.994 1.000 0.995 0.969

[0541] (0.950 / 0.020) (0.990 / 0.000) (0.994 / 0.000) (0.950 / 0.040) MP3 0.975 0.990 0.995 1.000 0.994 1.000 0.995 0.947

[0542] (0.970 / 0.020) (0.990 / 0.000) (0.994 / 0.006) (0.960 / 0.050) Pink 0.995 0.990 0.995 0.920 0.997 0.918 0.995 0.940 Noise (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.000) Random 0.995 1.000 0.995 1.000 0.997 1.000 0.995 0.980 Noise (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.980 / 0.000) Smooth 0.990 1.000 0.995 1.000 0.997 1.000 0.985 0.927

[0543] (0.980 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.020) Speed 0.905 0.000 0.995 0.000 0.994 0.000 0.935 0.000

[0544] (0.830 / 0.020) (0.990 / 0.000) (0.994 / 0.006) (0.920 / 0.050) Resample 0.970 1.000 0.995 1.000 0.994 1.000 0.960 0.949

[0545] (0.960 / 0.020) (0.990 / 0.000) (0.987 / 0.000) (0.940 / 0.020) AAC 0.990 0.960 0.995 1.000 0.997 1.000 0.980 0.950

[0546] (0.980 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.980 / 0.020) EnCodec 0.720 0.970 0.810 0.000 0.956 1.000 0.735 0.706 (nq=16) (0.460 / 0.020) (0.730 / 0.110) (0.943 / 0.031) (0.620 / 0.150) Crop 0.995 0.947 0.995 1.000 0.997 0.992 0.995 0.964

[0547] (0.990 / 0.000) (0.990 / 0.000) (0.994 / 0.000) (0.990 / 0.000)

[0548]

[0549] Average 0.967 0.926 0.983 0.870 0.994 0.932 0.965 0.888 (0.939 / 0.006) (0.974 / 0.007) (0.990 / 0.003) (0.952 / 0.023) Table 10. The performance of detection and attribution of the proposed method on different in-distribution datasets over different editing operations.

[0550] Dataset AudioMarkBench ASVspoof MusicGen MUSDB18 Edit Det. Att. Det. Att. Det. Att. Det. Att.

[0551] (TPR / FPR) (TPR / FPR) (TPR / FPR) (TPR / FPR) Identity 0.938 1.000 0.967 1.000 0.995 0.917 0.996 1.000

[0552] (0.875 / 0.000) (0.933 / 0.000) (0.990 / 0.000) (0.991 / 0.000) Bandpass 0.938 1.000 0.967 1.000 0.995 1.000 0.996 1.000

[0553] (0.875 / 0.000) (0.933 / 0.000) (0.990 / 0.000) (0.991 / 0.000) Boost 0.938 1.000 0.967 1.000 0.990 0.969 0.996 1.000

[0554] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.000) (0.991 / 0.000) Duck 0.938 1.000 0.967 1.000 0.990 0.969 0.996 1.000

[0555] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.000) (0.981 / 0.000) Echo 0.938 1.000 0.967 1.000 0.984 0.938 0.996 1.000

[0556] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.010) (0.991 / 0.000) Highpass 0.938 1.000 0.967 1.000 0.995 1.000 0.996 1.000

[0557] (0.875 / 0.000) (0.933 / 0.000) (0.990 / 0.000) (0.891 / 0.000) Lowpass 0.938 1.000 0.967 0.933 0.969 0.926 0.991 1.000

[0558] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.042) (0.991 / 0.009) MP3 0.938 1.000 0.933 0.867 0.969 0.926 0.991 1.000

[0559] (0.875 / 0.000) (0.867 / 0.000) (0.969 / 0.031) (0.983 / 0.000) Pink 0.938 1.000 0.967 1.000 0.990 0.979 0.996 1.000 Noise (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.000) (0.991 / 0.000) Random 0.938 1.000 0.967 1.000 0.990 0.896 0.996 1.000 Noise (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.000) (0.991 / 0.000) Smooth 0.938 1.000 0.967 0.867 0.974 0.895 0.996 1.000

[0560] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.031) (0.991 / 0.000) Speed 0.938 0.000 0.967 0.000 0.911 0.000 0.987 0.000

[0561] (0.875 / 0.000) (0.933 / 0.000) (0.896 / 0.073) (0.983 / 0.009) Resample 0.938 1.000 0.967 0.867 0.974 0.860 0.991 1.000

[0562] (0.875 / 0.000) (0.933 / 0.000) (0.979 / 0.031) (0.991 / 0.009) AAC 0.938 1.000 0.967 1.000 0.979 0.781 0.996 1.000

[0563] (0.875 / 0.000) (0.933 / 0.000) (0.990 / 0.031) (0.991 / 0.000) EnCodec 0.875 1.000 0.867 1.000 0.885 0.833 0.957 1.000 (nq=16) (0.750 / 0.000) (0.867 / 0.133) (0.875 / 0.104) (0.957 / 0.043) Crop 0.938 0.000 0.967 1.000 0.891 0.871 0.905 0.988

[0564] (0.875 / 0.000) (0.933 / 0.000) (0.875 / 0.094) (0.888 / 0.078)

[0565]

[0566] Average 0.934 0.875 0.958 0.908 0.967 0.860 0.986 0.937 (0.867 / 0.000) (0.925 / 0.008) (0.963 / 0.028) (0.982 / 0.009)

[0567]

[0568] Table 11. The performance of detection and attribution of the proposed method on different out-of-distribution datasets (unseen in training) over different editing operations.

[0569] C.7. Generalization Across Different Datasets

[0570] To investigate the performance of the proposed method across different audio durations and watermark strength a, experiments are conducted with audio duration ranging from Is to 10s and watermark strength a ranging from 0.0 to 1.5. As shown in Fig. 12, the present disclosure analyzes the impact of these parameters on detection and attribution accuracy. For duration analysis, it is observed that the detection accuracy remains robust (98.6-99.3%) across all tested durations from 1- 10s, demonstrating the proposed method’s effectiveness even with short audio clips. Attribution accuracy shows greater sensitivity, improving from 81.2% at Is to a peak of 93.0% at 5s, then stabilizing at 91.5-92.6% for longer durations. This suggests 5s segments provide the optimal trade-off between practical usability and attribution performance, though reliable detection (>98.5%) is achievable with clips as short as Is.

[0571] For watermark strength analysis, it is observed three distinct operational regimes: i) weak watermarks (a < 0.3), ii) moderate watermarks (0.4 < a < 0.7), and iii) strong watermarks (a > 0.8); iii) Strong watermarks (a > 0.8) achieves asymptotic performance with detection > 98.3% and attribution > 92.3%, showing diminishing returns beyond a = 1.0.

[0572] The default a = 1.0 configuration achieves 98.5% detection and 92.7% attribution accuracy. While higher a values (up to 1.5) marginally improve detection to 98.9%, attribution plateaus at 93.1-93.2%, suggesting an upper bound for useful watermark strength. Notably, the complete absence of watermarking (a = 0) yields chance-level detection (55.7%) and zero attribution capability, confirming the system’s dependence on the embedded signal.

[0573] C.8. Watermark Residual Visualization

[0574] To better understand the characteristics of different watermarking methods, the present disclosure visualizes the watermark residuals in Fig. 13. The visualization includes waveform, spectrogram, and mel- spectrogram representations for the original audio and four watermarking approaches (the proposed technique, AudioSeal, WavMark, and AudiowMark). The analysis reveals several key insights: (1) The proposed method introduces watermarks with the lowest energy, making them less perceptible in both time and frequency domains. (2) The spectral patterns of the proposed watermarks appear more stealthy and natural compared to other methods. (3) Unlike WavMark, which shows discrete interruptions, the proposed watermarking pattern exhibits better continuity, contributing to enhanced robustness against audio transformations while preserving imperceptibility.

[0575] C.9. Visualization of TF-weighted Penalty Distribution.

[0576] Comparison with TF-loudness loss in AudioSeal. AudioSeal adopts a coarse approach to implement auditory masking, using a naive loudness difference between the noise and the original audio. The main idea is that high loudness regions should mask lower loudness regions in the watermark residual. A positive penalty is assigned when the residual loudness is higher than the one of the original audio for each tile.

[0577] Specifically, the waveform is directly divided into B x W regions, where each region shares the same weighting term. Here, B represents the number of frequency bands for splitting, and W denotes the number of time tiles. The loudness difference between the watermark residual and the original waveform is then computed as I = Loudness(b^) — Loudness(x^), where w is the window index and b is the frequency band index. The weighting of the,, W\ penalty term for each tile is computed in a per-band soft-max manner, w =Bexp bThe ^d=iexP(^ ) final loudness loss is the weighted sum over all the tiles, fLoud

[0578]

[0579] = Sw=iSf=i • However, there are identified two limitations of this TF-based approach: (1) it lacks consideration of sophisticated auditory masking effects, where the masker and the maskee are not in the same tile; (2) using the loudness difference as the discrepancy measure only provides weak supervision for TF-guided watermarking in the spectrogram domain. To resolve these two issues, the present disclosure proposes to use a more sophisticated TF-weighted loss to guide the watermarking process. First, it is simulated in the TF domain the 2D energy asymmetric decay. This helps identify the potential masker-maskee pairs using filtering rules based on psychoacoustic principles. Secondly, instead of using loudness difference as a discrepancy measure, the present disclosure computes the mean-square error between the watermarked audio and the original one in the mel- spectrogram domain. This approach provides a more dense and fine-grained guidance signal for the imperceptible watermark.

[0580] Visualization of the TF-weighted penalty distribution. It is further visualized the penalty weight distribution of different TF-based approaches of AudioSeal and the proposed technique with the same spectrogram partitioning configuration. It is used a randomly sampled watermarked audio from a model that is at the beginning of training, where the watermark still did not exploit the potential of the TF masking effect for better imperceptibility. As can be seen from Fig. 14, the proposed method provides a more finegrained and dense penalty distribution, while AudioSeal’s approach just assigns the penalty weight to one tile (the red tile in the figure). Since the present disclosure uses multiple maskers in the vicinity to match their masked regions, the proposed method is able to provide broader and more dense penalty guidance. This indicates that the proposed method is more effective in exploiting the TF masking effect for better imperceptibility.

[0581] C.10. More Results on Watermark Localization

[0582] Similar to the Brute Force Detection (BFD) in WavMark, the present disclosure further extends the proposed model to have localization ability. Specifically, since the proposed model includes shifting-robust transformations and operates on Is segments, the per-segment detection probability can be distributed to the per-frame level with multiple overlapping detection windows.

[0583] Mathematically, let the audio signal be represented as a sequence of frames F =

[0584]

[0585] The proposed localization approach utilizes multiple overlapping detection windows, where each window

[0586]

[0587] corresponds to a Is audio segment. For each window

[0588]

[0589] the proposed model computes a watermark detection probability P(M|VI / k), where M denotes the presence of the watermark. A given frame ftmay be part of several such overlapping windows, let this set be denoted as W( j) =

[0590]

[0591] | ftG Wk}. The per-frame detection probability is then determined by aggregating the probabilities from all windows covering that frame. The average of P(M|VI / k) for all

[0592]

[0593] G W(f is taken as the per-frame detection probability i.e., P(M | / J = P(M | Wk). This process

[0594]

[0595] effectively distributes the segment-level detection scores to a finer, per-frame resolution, enabling precise localization of the watermarked regions.

[0596] The results are shown in Fig. 15. The experiments involve two setups: (1) fixing the audio length to 10s and varying the watermark segment length (2- 10s) within it; (2) fixing the watermark segment length to Is and varying the total audio length (2- 10s). For both setups, per-frame level detection is performed. Results show that the proposed technique can achieve comparable localization performance to AudioSeal and significantly outperforms WavMark. C.11. Sensitivity Test in Model Architecture

[0597] The present disclosure investigates the sensitivity of the proposed technique to key architectural choices, specifically the hidden dimension (H) of the message embedding table and the architecture of the temporal conditioning module. The results of this analysis, focusing on convergence speed for message decoding (message-bit validation accuracy), are presented in Fig. 16.

[0598] For the message embedding table, it is explored various hidden dimensions H relative to the secret message bit-length b (fixed at b = 16 in the experiments). A grid search is performed over H E {b / 2, b, 2b, 4b, 8b}. As illustrated in Fig. 16(a), the choice of H affects the convergence speed of the model, particularly for message decoding. It is found that H values ranging from b / 2 (i.e., 8) to 4b (i.e., 64) yield relatively fast convergence. However, using a much larger dimension, such as H = 8b (i.e., 128), can slow down the convergence process (the H = 128 case did not converge within 50k training steps).

[0599] Regarding the temporal conditioning module, an ablation study is conducted by comparing different MLP architectures: a linear projection (as proposed in the present disclosure), a 2-layer MLP, and a 3-layer MLP. The results, shown in Fig. 16(b), indicate that the linear projection is the most effective, being the only one among the tested options that converges successfully within the observed training steps. This suggests that the model’s convergence is sensitive to the architectural complexity of the temporal conditioning module.

[0600] C.12. Inference Efficiency and Model Size

[0601] The present disclosure additionally reports the model size, computation complexity, training speed, and inference speed comparison with AudioSeal in Table 12. The proposed model uses fewer parameters and has a smaller size for both the generator and the detector.

[0602] Although the proposed generator has higher FLOPs and a slightly increased inference time per segment (-0.3 ms / segment), the proposed detector significantly reduces FLOPs while maintaining similar inference efficiency overall. In training speed, the proposed model achieves a similar second-per- iteration rate (around 1.15 s / iter) as AudioSeal, with faster convergence speed in learning message decoding shown in Appendix C.2.

[0603] Method Component #Params Model FLOPs f Inference Time Train Speed 1 Size J, (ms) J, (s / iter) J,

[0604]

[0605] Generator 10.14M 38.67 1.18 x 0.411 ± 0.108 - MB 1011

[0606] Proposed Detector 7.59M 28.95 1.96 x 0.981 ± 1.57 - Technique MB 1011

[0607] Total 17.73M 67.62 3.14 x 1.39 + 1.58 1.15 + 0.51 MB 1011

[0608] Generator 14.68M 56.00 1.35 x 0.707 + 0.118 - MB 1010

[0609] Detector 8.65M 32.99 6.78 x 0.355 + 0.478 - AudioSeal

[0610] MB 1011

[0611] Total 23.33M 88.99 6.92 x 1.06 + 0.493 1.05 + 0.27 MB 1011

[0612]

[0613] Table 12. Comparison of model statistics and inference speed between the proposed method and AudioSeal. Training speed is computed with a batch size of 16 over the first 100 steps.

[0614] C.13. Comparison of Perceptual Quality under Different Watermark Strengths

[0615] To compare the perceptual quality of the proposed technique and AudioSeal under different watermark strengths, experiments are conducted adjusting the watermark intensity. For both the proposed technique and the AudioSeal baseline, Signal-to-Noise Ratios (SNRs) are targeted at levels from lOdB to 50dB, with interval of 5dB. The experimental results highlight two key advantages of the proposed approach. First, as illustrated in Fig. 17(a), the average attribution accuracy (across edits) of the proposed technique is consistently higher than AudioSeal when compared under different PESQ scores. Second, across different SNR levels, the proposed technique shows consistently better or comparable perceptual quality than AudioSeal in terms of PESQ scores. In a word, under different watermark strengths, the proposed technique achieves better / comparable performance in both attribution accuracy and perceptual quality, compared to AudioSeal.

[0616] C.14. Statistical Significance Analysis Across Transformations

[0617] To rigorously evaluate the performance differences in robustness between the proposed technique and AudioSeal across various audio transformations, statistical significance tests are conducted. McNemar’s test is employed to compare the proportions of correct attributions (a binary outcome for paired samples) and the Wilcoxon signed-rank test to compare the distributions of bit-wise message accuracy scores (a continuous measure for paired samples). In Table 13, the results of these tests are reported for each transformation applied to 10k audio samples per transformation. For the vast majority of transformations, the proposed technique demonstrates statistically significant improvements over AudioSeal. Specifically, the p-values for both McNemar’s test (on attribution correctness) and the Wilcoxon signed-rank test (on bit-wise accuracy) are typically far below the standard significance threshold of 0.05 (often p < 0.001, indicated as 0.000 in the table). This indicates that the superior performance of the proposed technique is not a result of random variation. For instance, in the comprehensive ‘Mixed of All’ scenario (consisting of all transformations), the proposed technique achieves an attribution accuracy of 0.88 compared to AudioSeal’s 0.58, and a bitwise accuracy of 0.99 versus 0.93 for AudioSeal. In a word, the proposed technique achieves statistically significant improvements over AudioSeal across various audio transformations.

[0618] McNemar’s Test Wilcoxon Test

[0619] T r ansformation

[0620] Proposed AudioSeal Significant Proposed AudioSeal Significant Encode (nq=16) 0.85±0.35 0.31±0.46 Yes (0.000) 0.99±0.03 0.85±0.15 Yes (9.7e-14) AAC compression 0.86±0.34 0.00±0.02 Yes (0.000) 0.99±0.03 0.62±0.10 Yes (2.8e-18) Bandpass filter 0.99±0.10 0.76±0.42 Yes (0.000) 1.00±0.00 0.98±0.05 Yes (7.1e-05) Boost audio 1.00±0.02 0.76±0.43 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (1.8e-05) Duck audio 1.00±0.02 0.78±0.42 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (1.2e-05) Echo 0.98±0.13 0.50±0.50 Yes (0.000) 1.00±0.00 0.93±0.11 Yes (6.8e-10) Highpass filter 0.99±0.10 0.77±0.42 Yes (0.000) 1.00±0.00 0.98±0.06 Yes (4.4e-05) Identity 1.00±0.01 0.78±0.42 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (5.9e-05) Lowpass filter 1.00±0.02 0.78±0.42 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (5.9e-05) MP3 compression 1.00±0.03 0.69±0.46 Yes (0.000) 1.00±0.00 0.98±0.05 Yes (1.4e-05) Pink noise 0.99±0.08 0.77±0.42 Yes (0.000) 1.00±0.01 0.98±0.04 Yes (1.7e-05) Random noise 1.00±0.01 0.76±0.43 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (3.6e-05) Smooth 1.00±0.02 0.54±0.50 Yes (0.000) 1.00±0.00 0.93±0.10 Yes (2.9e-10) Speed 0.03±0.17 0.00±0.05 Yes (7.8e-48) 0.43±0.08 0.53±0.14 Yes (9.5e-08) Up / down resample 1.00±0.02 0.78±0.42 Yes (0.000) 1.00±0.00 0.98±0.04 Yes (5.9e-05) Crop 0.76±0.43 0.37±0.48 Yes (0.000) 0.98±0.06 0.96±0.09 Yes (0.001) Mixed of All. 0.88±0.33 0.58±0.49 Yes (0.000) 0.99±0.07 0.93±0.13 Yes (0.000)

[0621]

[0622] Table 13. Statistical comparison of watermarking robustness between the proposed technique and AudioSeal on the main setup (user size of 104is used).

[0623] For each clip, 100 messages are generated and thus 10k audio samples in total for each transformation are obtained. Results show both McNemar’s test on attribution correctness (discrete values) and the Wilcoxon signed-rank test on bit-wise message accuracy (continuous values). Significance is marked as ‘Yes’ for p-values < 0.05 (i.e., a=0.05).

Claims

1. CLAIMS1. A neural network based system for use in audio watermarking, comprising:3.a neural network based watermark generator configured for embedding, based on a learnable embedding table, a predefined watermark into an input audio signal, thereby obtaining a watermarked audio signal; and4.a neural network based watermark detector comprising: a detection head that is configured for determining presence or absence of a watermark in a target audio signal; and a decoding head that is configured for decoding the watermark from the target audio signal, wherein the embedding table is shared between the watermark generator and the watermark detector, such that the decoding of the watermark from the target audio signal is based on the embedding table that has been used for the embedding.

2. The system according to claim 1, wherein the watermark comprises a binary message sequence.

3. The system according to claim 1 or 2, wherein the decoding head comprises a cross-attention module configured for decoding the watermark based on the shared embedding table.

4. The system according to any one of the preceding claims, wherein the watermark generator comprises an encoder module configured for transforming the input audio signal into a latent representation for the embedding.

5. The system according to claim 4, wherein the watermark generator further comprises a message encoder module configured for mapping the watermark to its latent representation, so as to enable embedding of the latent representation of the watermark into the latent representation of the input audio signal in the latent space, thereby obtaining a modulated latent representation of the input audio signal.

6. The system according to claim 5, wherein the latent representation of the watermark is obtained based on a temporal message conditioning mechanism that employs a linear modulation layer.

7. The system according to claim 5 or 6, wherein the watermark generator further comprises a decoder module configured for decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained; and wherein the decoder module shares a symmetric structure with the encoder module.

8. The system according to any one of the preceding claims, wherein the watermark detector comprises an encoder module configured for transforming the target audio signal into a reconstructed latent representation.

9. The system according to claim 8 when depending on claim 3, wherein the cross attention module is configured for decoding the watermark based on the reconstructed latent representation, by referencing the embedding table.

10. The system according to claim 9, wherein the embedding table is transformed into a key- value based reference based on linear projection for being used in the referencing.

11. The system according to claim 10, wherein the decoding of the watermark further involves attention-based vector merging with a query from the reconstructed latent representation.

12. The system according to any one of the preceding claims, wherein the neural network based system has been trained end-to-end based on at least one loss function.

13. The system according to claim 12, wherein the at least one loss function comprises a perceptual loss function.

14. The system according to claim 13, wherein the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency, TF, masking loss based on a psychoacoustic masking principle.

15. The system according to any one of the preceding claims, wherein the watermark detector is further configured to perform watermark localization based on a sliding window detection mechanism.

16. A method of training a neural network based system for use in audio watermarking,18.wherein the neural network based system comprises: a neural network based watermark generator for embedding a watermark into an input audio signal; and a neural network based watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal; and19.wherein the method comprises:20.obtaining, as a training input, a test audio signal and a test watermark sequence that are used as ground truth;21.embedding, by using the watermark generator and based on a learnable embedding table, the test watermark sequence into the test audio signal, thereby obtaining a watermarked audio signal;22.predicting, based on the watermarked audio signal, presence of a watermark by using the detection head, and an estimated watermark sequence by using the decoding head; and iteratively training the neural network based system based on at least one loss function indicative of respective difference between the predicted presence, the estimated watermark sequence, and the ground truth,23.wherein the method further comprises sharing the embedding table between the watermark generator and the watermark detector, such that the prediction of the estimated watermark sequence is based on the embedding table that has been used for the embedding.

17. The method according to claim 16, wherein the decoding head comprises a crossattention module; and wherein the estimated watermark sequence is predicted by the crossattention module based on the shared embedding table.

18. The method according to claims 16 or 17, wherein the watermark generator comprises an encoder module; and26.wherein the method further comprises, before the embedding:27.transforming, by using the encoder module, the test audio signal into a latent representation.

19. The method according to claim 18, wherein the watermark generator further comprises a message encoder module; and28.wherein the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal involves:29.mapping, by using the message encoder module, the test watermark sequence into its latent representation, so as to enable embedding of the latent representation of the test watermark sequence into the latent representation of the test audio signal in the latent space, thereby obtaining a modulated latent representation of the test audio signal.

20. The method according to claim 19, wherein the latent representation of the test watermark sequence is obtained based on a temporal message conditioning mechanism that employs a linear modulation layer.

21. The method according to claim 19 or 20, wherein the watermark generator further comprises a decoder module that shares a symmetric structure with the encoder module; and wherein the embedding, by using the watermark generator and based on an embedding table, the test watermark sequence into the test audio signal, in order to obtain a watermarked audio signal further comprises:32.decoding the modulated latent representation into a watermarked perturbation, based on which the watermarked audio signal is obtained.

22. The method according to any one of claims 16 to 21, wherein the watermark detector comprises an encoder module; and34.wherein the method further comprises, before the predicting:35.transforming, by using the encoder module, the watermarked audio signal into a reconstructed latent representation.

23. The method according to claim 22 when depending on claim 17, wherein the estimated watermark sequence is predicted by the cross attention module based on the reconstructed latent representation, by referencing the embedding table.

24. The method according to claim 23, wherein the method further comprises, before the prediction of the estimated watermark sequence, transforming the embedding table into a key- value based reference based on linear projection.

25. The method according to claim 24, wherein the prediction of the estimated watermark sequence further involves attention-based vector merging with a query from the reconstructed latent representation.

26. The method according to any one of claims 16 to 25, the at least one loss function comprises a perceptual loss function.

27. The method according to claim 26, wherein the perceptual loss function comprises a psychoacoustic-aligned temporal-frequency, TF, masking loss based on a psychoacoustic masking principle.

28. The method according to claim 27, wherein the psychoacoustic-aligned TF masking loss involves a weighted L2 loss in the Mel-spectrogram space, using a per-tile weight with asymmetric TF decay.

29. The method according to claim 27 or 28, wherein the psychoacoustic-aligned TF masking loss involves:42.identifying at least one masker region;43.applying TF decay around the masker region, so as to determine a masking threshold; determining at least one maskee region based on the masking threshold; and adjusting weights of the psychoacoustic-aligned TF masking loss, so as to encourage more watermark signal to be embedded in the maskee region.

30. The method according to any one of claims 16 to 29, wherein the method further comprises, before the predicting:45.applying at least one audio transformation to the watermarked audio signal to obtain a transformed audio signal; and46.feeding the transformed audio signal into the watermark detector.

31. The method according to claim 30, wherein the audio transformation comprises at least one of: compression, cropping, filtering, boost / duck, echo, noise addition, or artificial intelligence, Al, based technique.

32. The method according to claim 31, wherein applying the Al based transformation is based on a set of principal components in noise space obtained by principal component analysis, PC A.

33. The method according to any one of claims 16 to 32, wherein the iteratively training of the neural network based system comprises updating parameters of the watermark generator and / or the watermark detector based on evaluation of the at least one loss function.

34. A method of a neural network based system for audio watermarking, wherein the neural network based system comprises: a watermark generator for embedding a watermark into an input audio signal based on a learnable embedding table; and a watermark detector that comprises a detection head for determining presence of a watermark in a target audio signal and a decoding head for decoding the watermark from the target audio signal, wherein the embedding table is shared between the watermark generator and the watermark detector; and50.wherein the method comprises:51.obtaining the input audio signal;52.obtaining the watermark; and53.embedding, by using the watermark generator and based on the embedding table, the watermark into the input audio signal, thereby obtaining a watermarked audio signal.

35. The method according to claim 34, wherein the method further comprises: obtaining the target audio signal;55.determining, by using the detection head, presence or absence of a watermark in the target audio signal; and56.decoding the watermark from the target audio signal by using the decoding head based on the embedding table.

36. A method of a watermark detector for audio watermark detection and decoding, wherein the watermark detector comprises a detection head and a decoding head, and wherein the method comprises:58.obtaining a target audio signal;59.determining, by using the detection head, presence or absence of a watermark in the target audio signal; and decoding, by using the decoding head, the watermark from the target audio signal based on a learnable embedding table, wherein the embedding table has been used by a watermark generator during training for embedding a watermark into an input audio signal.

37. A watermark detector, comprising:61.a detection head configured for determining presence or absence of a watermark in a target audio signal; and62.a decoding head configured for decoding the watermark from the target audio signal based on a learnable embedding table, wherein the embedding table has been used by a watermark generator during training for embedding a watermark into an input audio signal.

38. An apparatus, comprising a processor and a memory coupled to the processor, wherein the processor is adapted to carry out the method according to any one of claims 16 to 36.

39. A program comprising instructions that, when executed by a processor, cause the processor to carry out the method according to any one of claims 16 to 36.

40. A computer-readable storage medium storing the program according to claim 39.

Images