Audio watermark processing method, apparatus, device, and medium

Abstract

The application relates to the field of artificial intelligence and audio processing technology, can be applied to the fields of intelligent medical treatment and financial technology, and discloses an audio watermark processing method, device, equipment and medium, which comprises the following steps: encoding audio blocks continuously input, obtaining an audio feature vector; acquiring a digital watermark, using a multilayer perceptron to time-modulate the digital watermark according to the audio length of the blocks, obtaining a watermark hidden vector; embedding the watermark hidden vector into the audio feature vector, obtaining a target audio vector; quantizing the target audio vector, generating a target audio spectrum embedded with the digital watermark; and decoding the target audio spectrum to obtain a target audio. The application encodes audio blocks continuously input, can adapt to a streaming scenario, processes audio data in sections, meets the watermark embedding requirement of continuous audio streaming, can compress audio, effectively reduces the data volume, reduces the transmission time consumption, and meets the low-delay communication requirement.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and audio processing technology, and in particular to an audio watermarking processing method, apparatus, device and medium. Background Technology

[0002] Voice digital watermarking technology verifies the authenticity of voice content by embedding imperceptible data watermarks into audio data. It can be used in areas such as audio copyright protection, covert communication, and detection of synthetic content. With the continuous advancement of artificial intelligence, the traditional financial and medical industries are gradually transforming towards technological and intelligent transformation. Voice digital watermarking technology is also increasingly being widely applied in content detection within fintech and smart healthcare. For example, in remote diagnosis scenarios in smart healthcare, patient voice is a crucial diagnostic basis. Doctors use patient voice content to assess a patient's health status. However, intelligent assistants used for remote diagnostic dialogues often transmit patient voice to doctors via public networks, posing cybersecurity risks. Embedding watermark information (patient identity information, doctor identity information, electronic medical record information) can detect whether the voice is complete and whether it has been injected with false medical information. In the financial sector, such as in telephone banking, conference recording, and identity verification scenarios, user voice commands involve sensitive information (such as transaction information and account information). Embedding watermark information (user identity information, transaction identifiers) can detect whether the voice has been tampered with (such as edited content or the insertion of false commands) and can trace the source of voice data leakage.

[0003] Currently, watermark embedding is usually performed by processing the entire audio file at once, which cannot adapt to network environments with high real-time requirements such as live streaming, where audio data is transmitted segment by segment. It is difficult to meet the watermark embedding requirements of dynamic and continuous audio streams. Moreover, in network transmission scenarios, in order to ensure the accuracy and integrity of the watermark, the amount of voice data after watermark embedding increases significantly, resulting in extended data processing and transmission time during transmission, which cannot meet the strict requirements of low-latency communication. Summary of the Invention

[0004] This invention provides an audio watermarking processing method, apparatus, device, and medium to solve the technical problems of poor adaptability and high latency of existing voice digital watermarking technology in streaming scenarios.

[0005] Firstly, an audio watermarking processing method is provided, including:

[0006] The continuously input audio blocks are encoded to obtain audio feature vectors;

[0007] A digital watermark is obtained, and a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks to obtain the watermark latent vector.

[0008] The watermark latent vector is embedded into the audio feature vector to obtain the target audio vector;

[0009] The target audio vector is quantized to generate a target audio spectrum embedded with a digital watermark;

[0010] The target audio is obtained by decoding the target audio spectrum.

[0011] Secondly, an audio watermarking processing device is provided, comprising:

[0012] The encoding module is used to encode continuously input audio blocks to obtain audio feature vectors;

[0013] The watermark processing module is used to acquire digital watermarks and uses a multilayer perceptron to perform time modulation on the digital watermarks according to the audio length of the blocks to obtain the watermark latent vector.

[0014] The watermark embedding module is used to embed the watermark latent vector into the audio feature vector to obtain the target audio vector.

[0015] The quantization module is used to quantize the target audio vector and generate the target audio spectrum with embedded digital watermark;

[0016] The decoding module is used to decode the target audio spectrum to obtain the target audio.

[0017] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described audio watermarking processing method.

[0018] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described audio watermarking processing method.

[0019] In the above-mentioned audio watermarking processing method, apparatus, device, and medium, a continuously input audio can be received by a client, and the continuously input audio can be divided into blocks and encoded to obtain an audio feature vector; a digital watermark can also be obtained, and a multilayer perceptron can be used to time-modulate the digital watermark according to the length of the audio blocks to obtain a watermark latent vector; then the watermark latent vector is embedded into the audio feature vector to obtain a target audio vector; and the target audio vector is quantized to generate a target audio spectrum embedded with a digital watermark; finally, the target audio spectrum is decoded to obtain the target audio. In this invention, for audio content detection in intelligent dialogue in real-time business scenarios such as smart healthcare and fintech, it is possible to... By embedding watermarks into the transmitted audio during real-time dialogue, the audio content can be detected subsequently through the watermark. Preferably, during the audio watermarking process of this invention, the continuously input audio can be encoded to compress the audio, effectively reducing the data volume and transmission time, meeting the requirements of low-latency communication. Moreover, the audio block encoding can adapt to streaming scenarios, processing audio data segment by segment to realize watermark generation and dynamic embedding, meeting the watermark embedding requirements of continuous audio streams. Furthermore, during watermark generation, a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks, obtaining a watermark latent vector that is co-distributed in the time and frequency domains, which can improve the hiding ability of the watermark, thereby improving the concealment and robustness of the watermark embedding. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application environment for an audio watermarking processing method according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart illustrating the audio watermarking processing method in the first embodiment of the present invention;

[0023] Figure 3 yes Figure 2 A schematic diagram of a specific implementation of step S120;

[0024] Figure 4 yes Figure 2 A schematic diagram of a specific implementation of step S130;

[0025] Figure 5 This is a flowchart illustrating the audio watermarking processing method in the second embodiment of the present invention;

[0026] Figure 6 yes Figure 5 A schematic diagram of a specific implementation of step S280;

[0027] Figure 7 This is a schematic diagram of the structure of an audio watermarking processing device in one embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0029] Figure 9 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] The audio watermarking method provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can receive continuously input audio from the client and encode the continuously input audio into blocks to obtain audio feature vectors; it can also acquire digital watermarks, use a multilayer perceptron to time-modulate the digital watermark according to the length of the audio blocks to obtain a watermark latent vector; then embed the watermark latent vector into the audio feature vector to obtain a target audio vector; and quantize the target audio vector to generate a target audio spectrum embedded with the digital watermark; finally, decode the target audio spectrum to obtain the target audio. In this invention, for audio content detection in real-time business scenarios such as smart healthcare and fintech, such as subsequent detection of the integrity of the call content and the authenticity of sensitive information (identity information, etc.), it can... By embedding watermarks into the transmitted audio during real-time dialogue, the audio content can be detected subsequently through the watermark. Preferably, during audio watermarking processing, the continuously input audio can be encoded to compress the audio, effectively reducing data volume and transmission time, meeting low-latency communication requirements, while improving audio processing quality and efficiency. Furthermore, audio block encoding adapts to streaming scenarios, processing audio data segment by segment to achieve watermark generation and dynamic embedding, meeting the watermark embedding requirements of continuous audio streams. During watermark generation, a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks, obtaining a watermark latent vector distributed in both the time and frequency domains, improving the watermark's concealment capability and thus enhancing the concealment and robustness of the watermark embedding. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0032] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the audio watermarking processing method provided in the first embodiment of the present invention includes the following steps S110-S150:

[0033] S110: Encode the continuously input audio blocks to obtain audio feature vectors.

[0034] In this step, the continuously input audio is segmented into consecutive blocks in chronological order. The encoder encodes the segmented audio to obtain audio feature vectors. In this embodiment, the encoder includes a 1D convolutional layer, B convolutional blocks connected to the 1D convolutional layer, and each convolutional block contains three residual units. The three residual units use dilated convolutions with dilation rates of 1, 3, and 9, respectively, to effectively capture features at different scales of the audio. After the convolutional blocks, a downsampling layer is connected, implemented in the form of stride convolution. Finally, a 1D convolutional layer is connected to output a feature vector of fixed dimension H.

[0035] In this invention, continuously input audio is encoded in segments, which can adapt to streaming scenarios, process audio data segment by segment, and compress audio, effectively reducing data volume, reducing transmission time, and meeting the requirements of low-latency communication.

[0036] The audio watermarking method provided by this invention can be applied to the detection of the integrity and authenticity of audio content in real-time business scenarios such as smart healthcare and fintech, typically implemented through the server side. For example, in remote diagnosis scenarios in the smart healthcare field, intelligent assistants usually transmit patient voice to doctors via public networks, posing network security risks. When sensitive voice information needs to be transmitted, such as the patient's dictated identity information, past medical history, and current condition, a digital watermark can be embedded in the call audio during real-time dialogue. The intelligent assistant and doctor can then use the digital watermark to detect whether the transmitted voice is complete and whether it has been tampered with to inject false medical information. Similarly, in the financial field, such as in financial telephone transactions where identity verification is required, the audio watermarking method of this invention can also be used to embed a watermark into the transaction voice information to detect the authenticity of the voice content.

[0037] S120: Obtain the digital watermark, and use a multilayer perceptron to perform time modulation on the digital watermark according to the audio length of the blocks to obtain the watermark latent vector.

[0038] Specifically, such as Figure 3 As shown, step S120 includes the following steps S121-S122:

[0039] S121: Use the watermark embedding table to obtain the watermark message vector based on the acquired digital watermark.

[0040] In this invention, the watermark embedding table includes the watermark embedding bit, embedding bit value, and embedding rules, etc. The digital watermark can be a randomly generated K-bit watermark, such as 0100010. The K-bit digital watermark is converted into an integer index, and the watermark message vector is obtained by looking up the index in the watermark embedding table.

[0041] S122: Using a multilayer perceptron, the watermark message vector is time-weighted and modulated according to the audio length of the blocks to obtain the watermark latent vector.

[0042] In this step, a multi-layer perceptron (MLP) is used to determine the audio length T of the segment using the formula h. w =W·V(w)∈R t′×H Time-weighted modulation is applied to the watermark message vector V(w) to obtain the watermark latent vector that is co-distributed in the time and frequency domains; where H is the dimension, W is the weight matrix of the multilayer perceptron, and W∈R K×t′ .

[0043] S130: Embed the watermark latent vector into the audio feature vector to obtain the target audio vector.

[0044] Specifically, such as Figure 4 As shown, step S130 includes the following steps S131-S132:

[0045] S131: Analyze the audio feature vector using a pre-trained embedding model based on the principle of psychoacoustic masking to obtain the masked region.

[0046] S132: Calculate the masking energy of the masked time-frequency block within the masked area, and embed the watermark hidden vector into the time-frequency block according to the magnitude of the masking energy to obtain the target audio vector.

[0047] In this step, weights are assigned to different time-frequency blocks based on their masking energy, and the watermark latent vector is embedded into the time-frequency blocks according to the weights to obtain the target audio vector. In this invention, time-frequency blocks with high masking energy can embed more watermarks without being detected, while time-frequency blocks with low masking energy can embed fewer watermarks or none at all, in order to avoid introducing audible distortion.

[0048] In this invention, the embedded model is trained using reconstruction loss and adversarial loss, and the overall loss function of the model is L = η·L. adv +L recon L adv The adversarial loss function is used, where η is a hyperparameter, which can be set to 0.1 during training. recon To reconstruct the loss function, the W-GAN (WassersteinGAN) method can be used with Wasserstein distance as the adversarial loss, and the reconstruction loss function is: and Among them, S W (m, t) represents the amplitude of the Mel spectrogram of the target audio, S O (m,t) represents the amplitude of the Mel-spectrum of the input audio, λ(m,t) represents the time-frequency block weighting term, and M masker This indicates that a set of time-frequency blocks suitable for watermark embedding has been selected, and in this embodiment, Mmasker ={(m,t)|S O (m,t)>α S max t′ S O (m,t′)},α S The amplitude threshold parameter can be 0.8, α S max t’ S O (m, t′) is the set threshold. Representing the global masking energy, in this embodiment, the maximum value of the masking energy in the time-frequency block set can be taken: Among them, E mask (m,t;(m c ,t c E represents the masking energy of the masked time-frequency block within the masked area. mask (m,t;(m c ,t c ))=20log 10 S O (m c ,t c )-ΔE, where ΔE is the attenuation considering both frequency and time directions:

[0049]

[0050] in, They respectively demonstrate different attenuation characteristics in the frequency and time directions.

[0051] For steps S131-S132, the audio is analyzed according to the psychoacoustic masking principle to identify the masked area. By calculating the masking energy, weights are assigned to different time-frequency blocks so that more watermark latent vectors can be embedded in the masked area with larger masking energy. This achieves the goal of effectively embedding the watermark into the area of ​​the audio that is not easily detected without affecting the audio perception quality, thereby improving the imperceptibility of the watermark and enhancing its robustness.

[0052] S140: Quantize the target audio vector to generate a target audio spectrum embedded with a digital watermark.

[0053] In this step, the target audio vector is quantized using the Residual Vector Quantization (RVQ) method to generate the target audio spectrum.

[0054] S150: Decode the target audio spectrum to obtain the target audio.

[0055] In this step, the target audio spectrum is decoded by a decoder to restore the audio. In this invention, the decoder in this step is symmetrical to the encoder structure used to encode the segmented audio. It first passes through a 1D convolutional layer, then upsamples through the transposed convolution of the upsampling layer, followed by B convolutional blocks, each containing three residual units, using a stride in the reverse order of the encoder to reconstruct the waveform, and finally passes through another 1D convolutional layer to output the target audio.

[0056] As can be seen, in the above solution, when detecting audio content of intelligent dialogue in real-time business scenarios such as smart healthcare and fintech, watermarking can be embedded into the transmitted audio during the real-time dialogue process, so that the audio content can be detected through the watermark later. Preferably, when processing audio watermarks, the continuously input audio can be encoded to compress the audio, effectively reducing the amount of data, reducing transmission time, and meeting the requirements of low-latency communication. Moreover, audio block encoding can adapt to streaming scenarios, processing audio data segment by segment to realize watermark generation and dynamic embedding, which meets the watermark embedding requirements of continuous audio streams. Furthermore, when generating the watermark, a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio block to obtain a watermark latent vector that is co-distributed in the time domain and frequency domain, which can improve the hiding ability of the watermark, thereby improving the concealment and robustness of the watermark embedding.

[0057] Please see Figure 5 As shown, Figure 5 A flowchart illustrating the audio watermarking processing method provided in the second embodiment of the present invention includes the following steps S210-S290:

[0058] S210: Encode the continuously input audio blocks to obtain audio feature vectors.

[0059] This step is the same as or similar to step S110 above, and will not be repeated here.

[0060] S220: Obtain the digital watermark, and use a multilayer perceptron to perform time modulation on the digital watermark according to the audio length of the blocks to obtain the watermark latent vector.

[0061] This step is the same as or similar to step S120 above, and will not be repeated here.

[0062] S230: Embed the watermark latent vector into the audio feature vector to obtain the target audio vector.

[0063] This step is the same as or similar to step S130 above, and will not be repeated here.

[0064] S240: Quantize the target audio vector to generate a target audio spectrum embedded with a digital watermark.

[0065] This step is the same as or similar to step S140 above, and will not be repeated here.

[0066] S250: Decode the target audio spectrum to obtain the target audio.

[0067] This step is the same as or similar to step S150 above, and will not be repeated here.

[0068] S260: Encode the target audio to obtain the watermark audio vector.

[0069] In this embodiment, a watermark detection process is added. In this step, the target audio with embedded digital watermark can be encoded by an encoder.

[0070] S270: Using a multilayer perceptron, the watermark embedding table is converted into a key sequence and a value sequence through linear projection.

[0071] In this step, the watermark embedding table is grouped according to two adjacent entries (indexed as 2*K and 2*K+1), and concatenated to form a 2H-dimensional vector. The concatenated vector is then subjected to two MLP linear projections (one for generating keys and one for generating values) to obtain key sequences and value sequences. Here, two adjacent entries represent two different bit states at the same position (e.g., 0 and 1). The keys are used to match and query the watermark audio vector, and the values ​​contain the watermark bit state information associated with these keys that needs to be extracted or used for embedding.

[0072] S280: Attention calculation is performed on the key sequence, value sequence and watermark audio vector through a cross-attention mechanism to obtain the bit state of the embedded bits in the watermark audio vector, thereby obtaining the digital watermark.

[0073] In this step, the watermark audio vector is demodulated to obtain a query sequence (a sequence containing feature representations of potential watermark information). Then, a cross-attention mechanism is used to detect the bit state of the embedded bits in the query sequence based on the key sequence and value sequence to obtain the digital watermark.

[0074] Specifically, such as Figure 6 As shown, in this embodiment, step S280 includes the following steps S281-S282:

[0075] S281: Demodulate the watermark audio vector using a multilayer perceptron to obtain a query sequence containing potential watermark information.

[0076] In this step, the potential watermark information contained in the predicted watermark audio vector is demodulated by MLP and converted into a set of learnable query vectors to obtain a query sequence. For example, the query sequence can be the probability of whether each bit predicted by demodulation contains watermark information. The length and dimension of the query sequence are compatible with the key / value sequence.

[0077] S282: Calculate the dot product similarity between each query vector and the key sequence in the query sequence based on the cross-attention mechanism to obtain the attention weight. Then, perform a weighted summation on the value sequence based on the attention weight to obtain the bit state of the embedding bit. Finally, obtain the digital watermark based on the bit state of all embedding bits.

[0078] In this step, the cross-attention formula is used. The attention weights are calculated, where Q is the query sequence, K is the key, and H is the dimension. The attention weights are weighted and summed with the values ​​to obtain the bit state of the watermark embedding bit. The obtained bit state is linearly projected onto the target dimension using MLP, and the watermark bit value is obtained through the sigmoid function and threshold decision, thereby decoding to obtain the embedded digital watermark.

[0079] S290: Use a pre-trained multilayer perceptron to classify the watermark audio vector, thereby identifying whether the target audio contains a watermark.

[0080] In this invention, to improve the watermark detection effect, a pre-trained MLP is used to quickly detect whether a watermark exists. In this step, the pre-trained MLP is used to classify the watermark audio vector to determine whether the target audio contains a watermark. If a watermark exists, it is determined as 1; if no watermark exists, it is determined as 0.

[0081] As can be seen, in the above solution, when detecting audio content in intelligent dialogues in real-time business scenarios such as smart healthcare and fintech, watermarking can be embedded into the transmitted audio during the real-time dialogue process. This allows for subsequent detection of audio content through the watermark. For example, in remote diagnosis scenarios, during the audio transmission watermark embedding stage, the intelligent assistant can use this invention to encode the continuously input audio blocks on the patient's end, adapting to streaming scenarios and processing audio data segment by segment. This meets the watermark embedding requirements of continuous audio streams and can compress audio, effectively reducing data volume and transmission time, thus meeting low-latency communication requirements. Furthermore, the watermark generation and embedding utilizes multilayer perceptron modulation, combined with reconstruction loss and adversarial loss optimization training, improving robustness while ensuring the watermark's imperceptibility. During the watermark detection stage, the intelligent assistant can use this invention on the doctor's end, combined with an attention mechanism, to accurately detect and decode the watermark, detecting whether the transmitted voice is complete and whether it has been tampered with to inject false medical information.

[0082] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0083] In one embodiment, an audio watermarking processing apparatus is provided, which corresponds one-to-one with the audio watermarking processing method in the second embodiment described above. For example... Figure 7 As shown, the audio watermarking processing device includes an encoding module 101, a watermarking processing module 102, a watermark embedding module 103, a quantization module 104, and a decoding module 105. Detailed descriptions of each functional module are as follows:

[0084] The encoding module 101 is used to encode continuously input audio blocks to obtain audio feature vectors;

[0085] The watermark processing module 102 is used to acquire a digital watermark and use a multilayer perceptron to perform time modulation on the digital watermark according to the length of the audio blocks to obtain the watermark latent vector.

[0086] The watermark embedding module 103 is used to embed the watermark latent vector into the audio feature vector to obtain the target audio vector;

[0087] Quantization module 104 is used to quantize the target audio vector to generate a target audio spectrum embedded with a digital watermark.

[0088] The decoding module 105 is used to decode the target audio spectrum to obtain the target audio.

[0089] In one embodiment, the watermark processing module 102 is specifically used for:

[0090] The watermark message vector is obtained based on the acquired digital watermark using the watermark embedding table;

[0091] The watermark message vector is time-weighted and modulated according to the audio length of the segments using a multilayer perceptron to obtain the watermark latent vector.

[0092] In one embodiment, the watermark embedding module 103 is specifically used for:

[0093] The pre-trained embedding model is used to analyze the audio feature vector based on the principle of psychoacoustic masking to obtain the masked region;

[0094] Calculate the masking energy of the masked time-frequency block within the masked area, and embed the watermark hidden vector into the time-frequency block according to the magnitude of the masking energy to obtain the target audio vector.

[0095] In one embodiment, the watermark embedding module 103 is further specifically used for: training during training using reconstruction loss and adversarial loss, wherein the reconstruction loss function is: and Among them, S W (m, t) represents the amplitude of the Mel spectrogram of the target audio, S O (m,t) represents the amplitude of the Mel-spectrum of the input audio, λ(m,t) represents the time-frequency block weighting term, and M masker This represents a set of time-frequency blocks that can be embedded with watermarks. This represents the global masking energy.

[0096] In one embodiment, the encoding module 101 is further configured to: encode the target audio to obtain a watermark audio vector;

[0097] The watermark processing module 102 is also used to: convert the watermark embedding table into a key sequence and a value sequence by using a multilayer perceptron through linear projection;

[0098] The decoding module 105 is also used to: perform attention calculation on the key sequence, value sequence and watermark audio vector through a cross-attention mechanism to obtain the bit state of the embedded bits in the watermark audio vector, thereby obtaining the digital watermark.

[0099] In one embodiment, the decoding module 105 is further configured to:

[0100] A pre-trained multilayer perceptron is used to classify the watermark audio vector, thereby identifying whether the target audio contains a watermark.

[0101] In one embodiment, the decoding module 105 is further specifically used for:

[0102] The watermark audio vector is demodulated using a multilayer perceptron to obtain a query sequence containing potential watermark information;

[0103] The dot product similarity between each query vector and the key sequence in the query sequence is calculated based on the cross-attention mechanism to obtain the attention weight. The value sequence is then weighted and summed according to the attention weight to obtain the bit state of the embedding bit. The digital watermark is obtained based on the bit state of all the embedding bits.

[0104] This invention provides an audio watermarking processing device that encodes continuously input audio blocks to obtain audio feature vectors; it can also acquire digital watermarks, using a multilayer perceptron to time-modulate the digital watermark according to the length of the audio blocks to obtain a watermark latent vector; then embeds the watermark latent vector into the audio feature vector to obtain a target audio vector; and quantizes the target audio vector to generate a target audio spectrum embedded with the digital watermark; finally, it decodes the target audio spectrum to obtain the target audio, thus realizing watermark embedding; furthermore, it can also obtain a watermark audio vector by encoding the target audio; using a multilayer perceptron to convert the watermark embedding table into a key sequence and a value sequence through linear projection; and then using a cross-attention mechanism to perform attention calculation on the key sequence, value sequence, and watermark audio vector to obtain... The bit state of the embedded bits in the watermark audio vector is used to obtain the digital watermark, which is then used for watermark detection. During operation, continuously input audio can be encoded to compress it, effectively reducing data volume and transmission time, meeting low-latency communication requirements. Audio block encoding adapts to streaming scenarios, processing audio data segment by segment to achieve watermark generation and dynamic embedding, satisfying the watermark embedding requirements of continuous audio streams. Furthermore, during watermark generation, a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks, obtaining a watermark latent vector distributed in both the time and frequency domains, improving the watermark's concealment capability and thus enhancing the concealment and robustness of watermark embedding. In the watermark detection stage, an attention mechanism is used to accurately detect and decode the watermark, significantly improving the performance of watermark detection and decoding.

[0105] Specific limitations regarding the audio watermarking processing device can be found in the limitations of the audio watermarking processing method described above, and will not be repeated here. Each module in the aforementioned audio watermarking processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0106] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of an audio watermarking processing method on the server side.

[0107] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the client-side functions or steps of an audio watermarking processing method.

[0108] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0109] The continuously input audio blocks are encoded to obtain audio feature vectors;

[0110] A digital watermark is obtained, and a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks to obtain the watermark latent vector.

[0111] The watermark latent vector is embedded into the audio feature vector to obtain the target audio vector;

[0112] The target audio vector is quantized to generate a target audio spectrum embedded with a digital watermark;

[0113] The target audio is obtained by decoding the target audio spectrum.

[0114] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0115] The continuously input audio blocks are encoded to obtain audio feature vectors;

[0116] A digital watermark is obtained, and a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks to obtain the watermark latent vector.

[0117] The watermark latent vector is embedded into the audio feature vector to obtain the target audio vector;

[0118] The target audio vector is quantized to generate a target audio spectrum embedded with a digital watermark;

[0119] The target audio is obtained by decoding the target audio spectrum.

[0120] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0122] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0123] The above-described embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention. Furthermore, any software tools or components not belonging to this company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

Claims

1. An audio watermarking processing method, characterized in that, include: The continuously input audio blocks are encoded to obtain audio feature vectors; A digital watermark is obtained, and a multilayer perceptron is used to time-modulate the digital watermark according to the length of the audio blocks to obtain the watermark latent vector. The watermark latent vector is embedded into the audio feature vector to obtain the target audio vector; The target audio vector is quantized to generate a target audio spectrum embedded with a digital watermark; The target audio is obtained by decoding the target audio spectrum; Specifically, embedding the watermark latent vector into the audio feature vector to obtain the target audio vector includes: The pre-trained embedding model analyzes the audio feature vectors based on the principle of psychoacoustic masking to obtain the masked region. The embedding model is trained using reconstruction loss and adversarial loss; the reconstruction loss function is: ,and ;in, The amplitude of the Mel spectrogram representing the target audio. This represents the amplitude of the Mel spectrogram of the input audio. Indicates the time-frequency block weighting term. This represents a set of time-frequency blocks that can be embedded with watermarks. Indicates global masking energy; Calculate the masking energy of the masked time-frequency block within the masked area, and embed the watermark hidden vector into the masked time-frequency block according to the magnitude of the masking energy to obtain the target audio vector.

2. The audio watermarking processing method as described in claim 1, characterized in that, The step of using a multilayer perceptron to perform time modulation on the digital watermark based on the length of the audio blocks to obtain the watermark latent vector includes: The watermark message vector is obtained based on the acquired digital watermark using the watermark embedding table; The watermark message vector is time-weighted and modulated according to the audio length of the segments using a multilayer perceptron to obtain the watermark latent vector.

3. The audio watermarking processing method as described in claim 1, characterized in that, The audio watermarking method further includes: The target audio is encoded to obtain the watermark audio vector; A multilayer perceptron is used to convert a watermark embedding table into a key sequence and a value sequence through linear projection. A cross-attention mechanism is used to perform attention calculations on the key sequence, value sequence, and watermark audio vector to obtain the bit state of the embedded bits in the watermark audio vector, thereby obtaining the digital watermark.

4. The audio watermarking processing method as described in claim 3, characterized in that, The audio watermarking method further includes: A pre-trained multilayer perceptron is used to classify the watermark audio vector, thereby identifying whether the target audio contains a watermark.

5. The audio watermarking processing method as described in claim 3, characterized in that, The process involves performing attention calculations on the key sequence, value sequence, and watermark audio vector using a cross-attention mechanism to obtain the bit state of the embedded bits in the watermark audio vector, thereby obtaining the digital watermark. This includes: The watermark audio vector is demodulated using a multilayer perceptron to obtain a query sequence containing potential watermark information; The dot product similarity between each query vector and the key sequence in the query sequence is calculated based on the cross-attention mechanism to obtain the attention weight. The value sequence is then weighted and summed according to the attention weight to obtain the bit state of the embedding bit. The digital watermark is obtained based on the bit state of all the embedding bits.

6. An audio watermarking processing device, characterized in that, include: The encoding module is used to encode continuously input audio blocks to obtain audio feature vectors; The watermark processing module is used to acquire digital watermarks and uses a multilayer perceptron to perform time modulation on the digital watermarks according to the audio length of the blocks to obtain the watermark latent vector. The watermark embedding module is used to embed the watermark latent vector into the audio feature vector to obtain the target audio vector. Specifically, it uses a pre-trained embedding model to analyze the audio feature vector based on the psychoacoustic masking principle to obtain the masked region. The embedding model is trained using reconstruction loss and adversarial loss, and the reconstruction loss function is: ,and ;in, The amplitude of the Mel spectrogram representing the target audio. This represents the amplitude of the Mel spectrogram of the input audio. Indicates the time-frequency block weighting term. This represents a set of time-frequency blocks that can be embedded with watermarks. Represents the global masking energy; calculates the masking energy of the masked time-frequency block within the masked region, and embeds the watermark hidden vector into the masked time-frequency block according to the magnitude of the masking energy to obtain the target audio vector; The quantization module is used to quantize the target audio vector and generate the target audio spectrum with embedded digital watermark; The decoding module is used to decode the target audio spectrum to obtain the target audio.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the audio watermarking processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the audio watermarking processing method as described in any one of claims 1 to 5.

Images